Reinforcement-Guided Synthetic Data Generation for Privacy-Sensitive Identity Recognition

Xuemei Jia; Jiawei Du; Hui Wei; Jun Chen; Joey Tianyi Zhou; Zheng Wang

cs.CVApr 9, 2026

Reinforcement-Guided Synthetic Data Generation for Privacy-Sensitive Identity Recognition

Authors: Xuemei Jia, Jiawei Du, Hui Wei, Jun Chen, Joey Tianyi Zhou, Zheng Wang

Organizations: National Engineering Research Center for Multimedia Software, School of Computer Science, Wuhan University, Wuhan, China · Hubei Key Laboratory of Multimedia and Network Communication Engineering, Wuhan, China · Centre for Frontier AI Research (CFAR) & Institute of High Performance Computing (IHPC), A*STAR, Singapore · Center for Machine Vision and Signal Analysis (CMVS), University of Oulu, Finland

Abstract

High-fidelity generative models are increasingly needed in privacy-sensitive scenarios, where access to data is severely restricted due to regulatory and copyright constraints. This scarcity hampers model development--ironically, in settings where generative models are most needed to compensate for the lack of data. This creates a self-reinforcing challenge: limited data leads to poor generative models, which in turn fail to mitigate data scarcity. To break this cycle, we propose a reinforcement-guided synthetic data generation framework that adapts general-domain generative priors to privacy-sensitive identity recognition tasks. We first perform a cold-start adaptation to align a pretrained generator with the target domain, establishing semantic relevance and initial fidelity. Building on this foundation, we introduce a multi-objective reward that jointly optimizes semantic consistency, coverage diversity, and expression richness, guiding the generator to produce both realistic and task-effective samples. During downstream training, a dynamic sample selection mechanism further prioritizes high-utility synthetic samples, enabling adaptive data scaling and improved domain alignment. Extensive experiments on benchmark datasets demonstrate that our framework significantly improves both generation fidelity and classification accuracy, while also exhibiting strong generalization to novel categories in small-data regimes.

Figures & tables

Figure 1: Pipeline comparison. (a) Existing methods rely solely on specific data, resulting in limited diversity and low utility of synthesized images. (b) We adapt broad, general-domain priors to the target domain, improving both diversity and task utility.

Types	Methods	Market-1501		CUHK03
Types	Methods	mAP	rank-1	mAP	rank-1
Base	ResNet-50 [ 13 ] †	$85.4$	$85.4$	$74.1$	$76.5$
Real Aug.	R-Erasing [ 66 ] † AAAI’20	$87.6$	$94.8$	$76.7$	$78.4$
	CIDAM [ 15 ] ACMMM’22	$87.4$	$95.1$	-	-
	CaAug [ 28 ] † ACMMM’24	$86.4$	$94.4$	$57.4$	$59.4$
Simulated Aug.	FineGPR [ 52 ] TOMM’23	$82.4$	$92.6$	$36.4$	$37.9$

Table 1: Comparisons with different synthesis-based SOTA methods on Market-1501 and CUHK03-NP. The comparison results are reproduced in our implementation to ensure fair and consistent evaluation. mAP(%) and Rank-1 (%) accuracy are reported.

Data Generation	Methods	LFW	AgeDB	CFP-FP	CA-LFW	CP-LWF	Avg.
Authentic	CASIA-Webface [ 54 ] subset	$91.58\%$	$74.72\%$	$76.00\%$	$78.78\%$	$71.15\%$	$78.47\%$
GAN	SFace [ 4 ] IJCB’22	$85.40\%$	$66.82\%$	$69.14\%$	$71.50\%$	$67.35\%$	$72.04\%$
	IDnet [ 24 ] CVPR’23	$85.53\%$	$68.73\%$	$69.91\%$	$72.67\%$	$68.12\%$	$73.00\%$
	SFace2 [ 3 ] T-BIOM’24	$85.58\%$	$68.12\%$	$69.26\%$	$72.35\%$	$66.83\%$	$72.43\%$
	DCFace [ 22 ] CVPR’23	$87.97\%$	$69.75\%$	$66.33\%$	$76.53\%$	$64.05\%$	$72.96\%$
DM	IDiff-Face [ 2 ] CVPR’23	$90.65\%$	$66.60\%$	$75.64\%$	$75.42\%$	$68.70\%$	7 $5.40\%$

Table 2: Comparison of on the proposed method with SOTA trained on small-scale CASIA-WebFace [ 54 ] subset. The highest and second-highest verification accuracies (%) are highlighted in red and blue , respectively.

Methods	Caucasian	Indian	Asian	African	Avg.
Authentic [ 54 ]	$72.85$	$70.10$	$65.45$	$60.28$	$67.17$
SFace [ 4 ]	$67.17$	$66.07$	$62.92$	$56.17$	$63.08$
IDnet [ 24 ]	$69.27$	$66.83$	$64.77$	$57.85$	$64.68$
DCFace [ 22 ]	$69.80$	$65.82$	$69.80$	$57.97$	$65.85$
SFace2 [ 3 ]	$67.78$	$66.28$	$63.68$	$58.22$	$63.99$
IDiff-Face [ 2 ]	$70.78$	$67.10$	$66.18$	$58.65$	$65.68$

Table 3: Demographic bias assessment of face recognition models trained with our method and SOTA approaches. The ethnicity-specific results report verification accuracies (%) on each subset of RFW [ 49 ] .

Figure 2: Comparisons with the baseline on Market-1501 generation. Real reference images are randomly selected from training set, where certain identity classes have only a few samples. While the baseline DiT benefits from external ImageNet pretraining to introduce moderate diversity, our RL-based fine-tuning further enhances intra-class variability, generating more diverse yet identity-consistent images.

Figure 3: Samples generated by our method. Real images are randomly selected from the training set of CASIA-WebFace, where certain identity classes have only a few available samples. While the baseline DiT benefits from external ImageNet pretraining to introduce moderate diversity, our RL-based fine-tuning generates more diverse images while largely preserving identity characteristics.

Figure 4: Feature distributions of real and synthesized samples across different methods. Image embeddings are extracted using a pretrained ResNet-50 and projected into a shared space via DOSNES [ 29 ] . Circles, squares, and triangles denote real samples, Random-Erasing synthesized samples, and synthesized samples generated by our method, respectively. Each color represents an identity class, with ten randomly selected classes visualized.

Figure 5: Ablation studies of our proposed method. Adding components consistently improve the face vertification accuracies (%).

Figure 6: Comparison of baseline models with and without our method across person ReID datasets.

Synthetic Data Generation

Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.

See all 623 papers

ScoreMix: Synthetic Data Generation by Score Composition in Diffusion Models Improves Recognition
Jun 11, 2025Parsa Rahimi, Sebastien MarcelSynthetic Data Augmentation Synthetic Data Generation
1EPFL, Switzerland · 2Idiap, Switzerland
Not All Similarity Is Leakage: Distribution-Aware Identity Leakage Assessment in Synthetic Face Datasets
Oct 19, 2026Paweł Borsukiewicz, Daniele Lunghi, Wendkûuni C. Ouédraogo +3Data Leakage Synthetic Data Evaluation
University of Luxembourg, Luxembourg · Fraunhofer IGD, Darmstadt, Germany
SteerFace: Debiasing Synthetic Face Generation via Adaptive Residue Perturbation
May 29, 2026Yuxi Mi, Qiuyang Yuan, Jianqing Xu +5Image Generation Identity-Preserving Image Generation
Fudan University Shanghai, China · Youtu Lab, Tencent Shanghai, China · WeChat Pay Lab33, Tencent Shenzhen, China

Jun 11, 2025J/K move · Enter open · S save

ScoreMix: Synthetic Data Generation by Score Composition in Diffusion Models Improves Recognition

Parsa Rahimi, Sebastien Marcel

1EPFL, Switzerland · 2Idiap, Switzerland

Synthetic Data Augmentation Synthetic Data Generation Face Recognition Score-Based Generative Modeling Diffusion Models Diffusion-Based Data Augmentation

Open paper AskPDF arXiv

Synthetic data generation is increasingly used in machine learning for training and data augmentation. Yet, current strategies often rely on external foundation models or datasets, whose usage is restricted in many scenarios due to policy or legal constraints. We propose ScoreMix, a self-contained synthetic generation method to produce hard synthetic samples for recognition tasks by leveraging the score compositionality of diffusion models. The approach mixes class-conditioned scores along reverse diffusion trajectories, yielding domain-specific data augmentation without external resources. We systematically study class-selection strategies and find that mixing classes distant in the discriminator's embedding space yields larger gains, providing up to 3% additional average improvement, compared to selection based on proximity. Interestingly, we observe that condition and embedding spaces are largely uncorrelated under standard alignment metrics, and the generator's condition space has a negligible effect on downstream performance. Across 8 public face recognition benchmarks, ScoreMix improves accuracy by up to 7 percentage points, without hyperparameter search, highlighting both robustness and practicality. Our method provides a simple yet effective way to maximize discriminator performance using only the available dataset, without reliance on third-party resources. Paper website: https://parsa-ra.github.io/scoremix/.

Abstract

Synthetic Data Generation

Explore similar work

ScoreMix: Synthetic Data Generation by Score Composition in Diffusion Models Improves Recognition

Not All Similarity Is Leakage: Distribution-Aware Identity Leakage Assessment in Synthetic Face Datasets

SteerFace: Debiasing Synthetic Face Generation via Adaptive Residue Perturbation