IDperturb: Enhancing Variation in Synthetic Face Generation via Angular Perturbation

Fadi Boutros; Eduarda Caldeira; Tahar Chettaoui; Naser Damer

cs.CVFeb 21, 2026

IDperturb: Enhancing Variation in Synthetic Face Generation via Angular Perturbation

Authors: Fadi Boutros, Eduarda Caldeira, Tahar Chettaoui, Naser Damer

Organizations: Fraunhofer Institute for Computer Graphics Research IGD, Germany · TU Darmstadt, Germany

Abstract

Synthetic data has emerged as a practical alternative to authentic face datasets for training face recognition (FR) systems, especially as privacy and legal concerns increasingly restrict the use of real biometric data. Recent advances in identity-conditional diffusion models have enabled the generation of photorealistic and identity-consistent face images. However, many of these models suffer from limited intra-class variation, an essential property for training robust and generalizable FR models. In this work, we propose IDPERTURB, a simple yet effective geometric-driven sampling strategy to enhance diversity in synthetic face generation. IDPERTURB perturbs identity embeddings within a constrained angular region of the unit hyper-sphere, producing a diverse set of embeddings without modifying the underlying generative model. Each perturbed embedding serves as a conditioning vector for a pre-trained diffusion model, enabling the synthesis of visually varied yet identity-coherent face images suitable for training generalizable FR systems. Empirical results demonstrate that training FR on datasets generated using IDPERTURB yields improved performance across multiple FR benchmarks, compared to existing synthetic data generation approaches.

Figures & tables

Figure 1: Identity perturbation in the embedding space. (a) a normalized reference embedding $\mathbf{v}\in\mathbb{R}^{3}$ . (b) random $\mathbf{n_{1}},\mathbf{n_{2}}\sim\mathcal{N}(0,\mathbf{I})$ sampled and projected onto the hyperplane orthogonal to $\mathbf{v}$ , resulting in $\mathbf{u_{1}}$ and $\mathbf{u_{2}}$ (Equation 4 ). (c) perturbed identity vector $\mathbf{\tilde{v}_{1}}$ and $\mathbf{\tilde{v}_{2}}$ with different angle to $\mathbf{v}$ are constructed (Equation 5 ). Note, this figure is not manually plotted, rather, using a Python script corresponding to Algo. 1 . The cone boundary, dashed circumference in (c), is defined by the lower bound $\mathbf{lb}$ is 0.66, and $\mathbf{n_{1}}$ and $\mathbf{n_{2}}$ are sampled on the fly to illustrate a real scenario. For visualization purposes, we set the number of dimensions to 3, $\mathbf{v}\in\mathbb{R}^{3}$ .

Figure 2: Effect of IDperturb (via lower bound $\mathbf{lb}$ ) on image diversity. The first row varies only the noise seed with fixed condition $\mathbf{v}$ , subsequent rows use perturbed identity embeddings $\mathbf{\tilde{v}}$ from $\mathbf{v}$ with decreasing $\mathbf{lb}$ (0.9 to 0.4). Lower $\mathbf{lb}$ values increase angular deviation, enhancing intra-class variation. Note that using low lb of e.g., 0.4, the identity of some samples could be less consistent, which is also quantified by high EER in Table 1 .

		Identity-Separability of Training Data							FR Verification
		Operation Metrics		Score Distributions					Verification Benchmarks
		Operation Metrics		genuine		imposter			Verification Benchmarks
Data Sampling Method	$\mathbf{lb}$	EER $\downarrow$	FMR100 $\downarrow$	G-mean	G-std	I-mean	I-std	FDR $\uparrow$	LFW $\uparrow$	AgeDB-30 $\uparrow$	CFP-FP $\uparrow$	CALFW $\uparrow$	CP-LFW $\uparrow$	Average $\uparrow$
C-WF [ 49 , 4 ]	-	0.076	0.092	0.536	0.215	0.003	0.070	5.541	99.55	94.55	95.31	93.78	89.95	94.63
Baseline (FFHQ)	-	0.005	0.004	0.509	0.104	0.023	0.081	13.680	97.60	84.10	83.36	89.08	78.77	86.58
IDperturb	0.9	0.010	0.011	0.448	0.108	0.023	0.060	11.817	98.05	86.78	83.79	89.92	79.02	87.51

Table 1: Identity separability of datasets (left side) and verification accuracy (%) of FR models trained on 0.5m images of each of those datasets (right side). The data was generated by varying the

d

-dimensional cone boundary

\mathbf{lb}\in[0.4,0.9]

on two baseline LDM models: IDiff-Face trained on FFHQ and IDiff-Face trained on C-WF. The FR results demonstrated that IDperturb achieves consistent and superior performance over the baselines across all FR benchmarks. The results of the first row were obtained from authentic C-WF.

Figure 3: FR average accuracies (as in Table 1 ) with respect to the intra-class consistency and diversity of baseline (C-WF) and IDPerturb of $lb\in[0.4,0.5,0.6,0.7,0.8,0.9]$ .

			Attributes					D-Intra	C-Intra	FR-Perf. $\uparrow$
Dataset	lb	scale	Age $\uparrow$	Pose $\uparrow$			Exp. $\uparrow$
				Yaw	Pitch	Roll
C-WF	-	-	0.354	23.479	9.069	5.154	0.589	0.423	0.948	94.63
Baseline (C-WF)	-	1	0.283	18.881	7.992	3.230	0.429	0.366	0.999	91.25
IDperturb	0.9	1	0.325	19.907	8.384	3.606	0.492	0.393	0.998	92.68
	0.8	1	0.369	20.899	8.727	4.029	0.538	0.417	0.994	93.31

Table 2: Intra-class diversity and consistency on top of the Baseline(C-WF) reported as age entropy, facial expression (Exp.) entropy, head-pose STD,

D_{intra}

and

C_{intra}

. Higher age and expression entropy, pose STD, and

D_{intra}

indicate greater intra-class diversity. The first row shows authentic C-WF results. Baseline (C-WF) is generated from DM without IDperturb . The first part of the table shows results of different

lb

and the second part presents the results of fixed

lb=0.6

and different scale

\omega

of CFG.

		Identity-Separability of Training Data							FR Verification
		Operation Metrics		Score Distributions					Verification Benchmarks
		Operation Metrics		genuine		imposter			Verification Benchmarks
Data Sampling Method	Scale	EER $\downarrow$	FMR100 $\downarrow$	G-mean	G-std	I-mean	I-std	FDR $\uparrow$	LFW $\uparrow$	AgeDB-30 $\uparrow$	CFP-FP $\uparrow$	CALFW $\uparrow$	CP-LFW $\uparrow$	Average $\uparrow$
IDperturb	0	0.162	0.427	0.185	0.124	0.006	0.058	1.718	99.25	92.58	92.31	93.53	87.17	92.97
	1	0.063	0.134	0.338	0.162	0.008	0.059	3.664	99.40	93.20	93.61	93.50	88.37	93.62
	2	0.043	0.079	0.394	0.168	0.012	0.060	4.611	99.33	93.33	93.56	93.42	88.53	93.63

Table 3: Identity separability of synthetic datasets (left side) and verification accuracy (%) of FR models trained on those datasets (right side). We compare IDperturb -based synthetic datasets (generated using base DM trained on C-WF with varying scale

\omega\in[0,5]

values and fixed

d

-dimensional cone boundary

\mathbf{lb}=0.6

Figure 4: Top: reference identity, first row is generated by Arc2Face and last row is generated by Arc2Face w/ IDPerturb .

	LFW	AgeDB-30	CFP-FP	CA-LFW	CP-LFW	Avg
Arc2Face	98.90	87.50	90.58	89.37	82.20	89.71
Arc2Face w/ IDPERTURB	98.93	89.81	92.57	90.90	84.40	91.32

Table 4: Verification accuracy (%) of FR models trained on data generated by Arc2Face and Arc2Face w/ IDPERTURB.

Method	Data Generation	DGMs Train Dataset	#imgs (IDs × imgs/ID)	LFW	AgeDB	CFP-FP	CA-LFW	CP-LFW	Avg	IJB-C
Method	Data Generation	DGMs Train Dataset	#imgs (IDs × imgs/ID)	LFW	AgeDB	CFP-FP	CA-LFW	CP-LFW	Avg	$10^{-5}$	$10^{-4}$
C-WF [ 49 , 4 ]	Authentic	-	0.49M( $\sim$ 10.5k × 47)	99.55	94.55	95.31	93.78	89.95	94.63	93.96	96.05
DigiFace [ 1 ] WACV’22	Digital Rendering	-	0.5M(10k × 50)	95.40	76.97	87.40	78.62	78.87	83.45	-	-
DigiFace [ 1 ] * WACV’22		-	0.5M(10k × 50)	91.15	74.00	82.93	75.30	73.40	79.36	30.01	44.78
DigiFace [ 1 ] WACV’22		-	1.2M(10k × 72 + 100k × 5)	96.17	81.10	89.81	82.55	82.23	86.37	-	-
IDnet [ 27 ] CVPRW’23	GAN-Based	C-WF	0.53M(10.5k × 50)	92.58	73.53	75.40	79.90	74.25	79.13	38.85	53.25

Table 5: FR verification accuracies of IDperturb and SOTA. We highlight the datasets used to train DGMs, FFHQ [ 24 ] (70k images) vs. C-WF [ 49 ] (0.49M images) vs. WebFace4M (WF4M) [ 53 ] (4M images). Under the same settings, i.e., DGM trained on large C-WF, IDperturb outperformed all competitors on the considered evaluation benchmarks. The results marked with a star (*) are reproduced in this work, as there was no publicly released FR model. Some of the previous works did not release their pre-trained model and their generated data and there were no reported results on the large-scale IJB-C [ 29 ] . The first row presents the results achieved by authentic C-WF, which are provided as a reference. Best results for each DM setup are shown in bold , and second-best ones are underlined .

Synthetic Data Generation

Synthetic data generation creates artificial examples, simulations, and datasets to train, test, or protect machine-learning systems.

See all 623 papers

SteerFace: Debiasing Synthetic Face Generation via Adaptive Residue Perturbation
May 29, 2026Yuxi Mi, Qiuyang Yuan, Jianqing Xu +5Image Generation Identity-Preserving Image Generation
Fudan University Shanghai, China · Youtu Lab, Tencent Shanghai, China · WeChat Pay Lab33, Tencent Shenzhen, China
Identity-Conditioned Latent Consistency Distillation for Face Synthesis
Aug 31, 2026Tiago Kienen Chaves, Bernardo Biesseck, David MenottiSynthetic Data Augmentation Latent Diffusion Models
Department of Informatics, Federal University of Paraná, Curitiba, Brazil · Federal Institute of Mato Grosso (IFMT), Pontes e Lacerda, MT, Brazil
Adaptive Adversarial Augmentation for Controllable Face Synthesis
Oct 8, 2026Saransh Suri, Shivang Agarwal, Mayank Vatsa +1Synthetic Data Augmentation Data Augmentation
Manipal University Jaipur, India · Indian Institute of Technology Jodhpur, India · Birla Institute of Technology and Science Pilani Dubai Campus, Dubai, United Arab Emirates

May 29, 2026J/K move · Enter open · S save

SteerFace: Debiasing Synthetic Face Generation via Adaptive Residue Perturbation

Yuxi Mi, Qiuyang Yuan, Jianqing Xu, Yichun Zhou+4

Fudan University Shanghai, China · Youtu Lab, Tencent Shanghai, China · WeChat Pay Lab33, Tencent Shenzhen, China

Image Generation Identity-Preserving Image Generation Diffusion-Based Image Generation Face Recognition

Open paper AskPDF arXiv

The shortage of legally compliant data for face recognition training has sparked growing interest in using synthetic data as an alternative. While recent diffusion-based methods enable the generation of photorealistic face images with strong identity adherence and data diversity, their downstream recognition performance still exhibits a significant synthetic-real gap. This paper identifies visual tendency as a previously underexplored limitation, whereby synthetic data exhibit an unrealistic prevalence of visual attributes and thus deviate from the real-data distribution. Visual tendency can be attributed to the generator's conditioning on identity embeddings, through which co-occurring residual visual cues are unintentionally absorbed into learned identity semantics. To discourage the generator from exploiting such visual cues, this paper proposes SteerFace, a simple and efficient training framework that perturbs identity embeddings by steering them toward random orthogonal directions on the embedding hypersphere. The perturbation serves as an identity-preserving regularizer that penalizes the generator's reliance on non-identity components, as supported by theoretical analysis. This paper further introduces an adaptive strategy that learns perturbation strengths with both sample-wise preference and favorable overall statistics. Extensive experiments show that SteerFace effectively mitigates visual tendency, outperforms prior methods in downstream face recognition, and generalizes well across different training datasets and generation pipelines.

Abstract

Synthetic Data Generation

Explore similar work

SteerFace: Debiasing Synthetic Face Generation via Adaptive Residue Perturbation

Identity-Conditioned Latent Consistency Distillation for Face Synthesis

Adaptive Adversarial Augmentation for Controllable Face Synthesis