IDPrior: Decoupling Identity Generation from Facial Image Synthesis

Fadi Boutros; Meriç Demirörs; Darian Tomašević; Vitomir Štruc; Naser Damer

cs.CVNov 5, 2026

IDPrior: Decoupling Identity Generation from Facial Image Synthesis

Authors: Fadi Boutros, Meriç Demirörs, Darian Tomašević, Vitomir Štruc, Naser Damer

Organizations: Fraunhofer Institute for Computer Graphics Research IGD, Germany · University of Ljubljana, Slovenia · TU Darmstadt, Germany

Abstract

Large-scale identity-labeled face datasets are essential for training modern face recognition models, yet collecting and annotating such data is costly and raises privacy and ethical concerns. Existing synthetic face generation methods typically derive identities from predefined class labels, unconditional image generation, or predefined latent sampling strategies, without explicitly modeling the underlying distribution of facial identities. In this paper, we propose IDPrior, a two-stage framework that decouples identity generation from image synthesis. We first learn a diffusion prior in the embedding space of a pretrained face recognition model to model the distribution of facial identities and generate novel identity embeddings. These embeddings are then used to condition a pretrained identity-conditioned diffusion model (DM), enabling the synthesis of multiple images of the same identity. This modular design provides a scalable approach to identity generation while remaining compatible with existing and future identity-conditioned DMs. Experiments indicate that IDPrior generates diverse and well-separated identities while maintaining a closer correspondence to the distribution of authentic identities. Face recognition models trained on datasets generated using IDPrior-generated identities achieve competitive or improved performance compared with existing synthetic-data-based methods across multiple face verification benchmarks. Code and generated datasets are publicly available at https://github.com/fdbtrs/IDperturb.

Figures & tables

Figure 1: Overview of IDPrior, a dual diffusion framework that decouples identity generation from image synthesis. Stage 1: A diffusion prior is learned on FR embeddings to model the distribution of facial identities and generate novel identity embeddings. Stage 2: The sampled embeddings are subsequently used to condition a pretrained identity-guided image diffusion model, enabling the synthesis of multiple images per identity.

Figure 2: Identity uniqueness as a function of the cosine-similarity threshold. We report the percentage of unique identities among 10K generated identities. As the threshold becomes stricter, IDPrior (DDPM) consistently preserves a larger fraction of unique identities than existing approaches.

Panel of Figure 3 — Figure 3: Number of well-separated identities as the total number of generated identities increases, using a threshold of 0.30.

Approach	#Identities	Unique in (%) @ $r$
Approach	#Identities	0.25	0.3	0.4
Random (uniform)	10K	99.98	100.00	100.00
	100K	99.98	100.00	100.00
	500K	99.88	100.00	100.00
	1M	99.77	100.00	100.00
VIGFace [ 24 ]	10K	100.00	100.00	100.00

Table 1: Scalability of identity generation from 10K to 1M identities. We report the percentage of unique identities at thresholds 0.25, 0.30, and 0.40. IDPrior (DDPM) maintains substantially higher uniqueness than PCA- and image-generation-based approaches across scales, while random sampling, VIGFace, and the flow-based prior achieve higher separation at the cost of poorer distribution fidelity, as detailed in Table 2 .

Method	Unique $@$ 0.3 $\uparrow$	Unique $@$ 0.25 $\uparrow$	ID-FID $\downarrow$	U-ID-FID $\downarrow$	Train Sim. $\downarrow$	$P_{95}$	$P_{99}$	Leak $@$ 0.25 (%)	Leak $@$ 0.3 (%)	Leak $@$ 0.4 (%)
	Unique		ID-FID		Novelty	Identity leakage $\downarrow$
CW (Authentic)	61.50	16.67	-	0.0309	-	-	-	-	-	-
WF4M (Authentic)	70.83	22.68	-	0.0560	-	-	-	-	-	-
Random (Uniform)	100.00	99.98	0.6009	0.3210	0.1666	0.1932	0.2075	0.00	0.00	0.00
VIGFace	100.00	100.00	0.9166	0.9987	0.1605	0.1903	0.2052	0.00	0.00	0.00
Arc2Face /Vec2Face	99.15	74.35	0.6759	0.6967	0.1686	0.1936	0.2087	0.01	0.00	0.00

Table 2: Identity uniqueness, distribution fidelity, and potential identity leakage on 10K generated identities. We report uniqueness at thresholds of 0.30 and 0.25, ID-FID, unified ID-FID (U-ID-FID) using a common CelebA [ 30 ] reference distribution, and nearest-training-identity similarity statistics, including the

95

th and

99

th percentiles (

P_{95}

and

P_{99}

) and leakage rates at

\tau

of 0.25, 0.3, and 0.4. Lower

P_{95}

P_{99}

, and leakage rates indicate lower similarity to training identities and therefore greater identity novelty, while lower ID-FID and U-ID-FID indicate better distributional fidelity.

Figure 4: Identity leakage at different thresholds. We report the percentage of leaked identities among 10K generated identities.

Face Embeddings	DGM	LFW	AgeDB-30	CFP-FP	CA-LFW	CP-LFW	Avg
Uncond. Samples	IDiff-Face [ 4 ]	98.75	88.85	91.61	90.90	86.15	91.25
Random (Uniform)	IDiff-Face [ 4 ]	98.00	87.30	90.97	91.25	86.01	90.71
VIGFace	IDiff-Face [ 4 ]	99.82	87.48	91.39	90.07	86.53	91.06
IDPrior(VAE)	IDiff-Face [ 4 ]	98.60	88.72	91.53	91.45	87.27	91.51
IDPrior (Flow-model)	IDiff-Face [ 4 ]	99.03	90.95	92.30	91.90	87.03	92.24
IDPrior (DDPM)	IDiff-Face [ 4 ]	98.82	90.80	92.67	91.95	87.07	92.26

Table 3: Comparison of identity generation strategies for training FR models. Verification accuracy (%) obtained using different identity sources with identity-conditioned DMs. Replacing the original identity sampling mechanisms with IDPrior generally improves downstream recognition performance across the evaluated identity-conditioned generators, with the largest gains observed for IDiff-Face and IP-Adapter-FaceID. Results marked with * are reproduced in this work.

	IDs × imgs/ID	LFW	AgeDB-30	CFP-FP	CA-LFW	CP-LFW	Avg
IDPrior	50K (1k x 50)	94.83	79.82	79.07	84.20	73.80	82.34
IDPrior	0.5M (10k x 50)	98.82	90.80	92.67	91.95	87.07	92.26
IDPrior	1.0M (20k x 50)	99.27	92.28	94.03	92.98	88.90	93.49
IDPrior	2.0M (40k x 50)	99.40	93.72	94.79	93.93	90.20	94.41
IDPrior	4.0M (80k x 50)	99.52	94.83	95.83	94.48	90.98	95.13

Table 4: FR performance trained on IDPrior (DDPM)-generated datasets of increasing scale. Performance generally improves, approaching the accuracy achieved using authentic training data.

Method	#imgs (IDs × imgs/ID)	LFW	AgeDB-30	CFP-FP	CA-LFW	CP-LFW	Avg
CW [ 49 , 4 ]	0.49M( $\sim$ 10.5k IDs)	99.55	94.55	95.31	93.78	89.95	94.63
WF4M [ 52 ]	4M( $\sim$ 206K IDs)	99.80	97.80	99.03	96.03	94.27	97.39
DigiFace [ 1 ] WACV’22	0.5M(10k × 50)	95.40	76.97	87.40	78.62	78.87	83.45
IDnet [ 27 ] CVPRW’23	0.53M(10.5k × 50)	92.58	73.53	75.40	79.90	74.25	79.13
SFace [ 3 ] IJCB’22	0.63M(10.5k × 60)	91.87	71.68	73.86	77.93	73.20	77.71
SFace2+ [ 8 ] TBIOM’24	0.63M(10.5k × 60)	95.60	77.37	77.11	83.40	74.60	81.62

Table 5: FR verification accuracies of IDPrior and existing synthetic-data-based approaches. At 0.5M and 1M images, IDPrior achieves competitive or SOTA performance among the compared methods. The first and second rows report authentic CW and WF4M as references. Under each setup, the best results are in bold and the second-best ones are underlined . Results in the last block are obtained by augmenting authentic CW [ 49 ] with synthetic data, following VIGFace [ 24 ] . Results marked with †are obtained from [ 24 ] .

	IJCB - TPR@FPR
	1e-5	1e-4	1e-3	1e-2
CW	61.41	85.32	92.71	96.73
DigiFace	-	-	30.01	44.78
IDNet	-	-	38.85	83.25
ExFaceGAN	-	-	12.92	73.28
IDiff-Face	-	-	23.44	69.69

Table 6: Verification results on the IJC-B benchmark.

Panel of Figure 5 — Figure 5: Scalability analysis under different training configurations. The number of well-separated identities vs. the total number of generated identities increases. We compare IDPrior trained on embeddings extracted by ArcFace and ElasticFace, as well as IDPrior trained on embeddings from WebFace4M, CASIA-WebFace, and FFHQ. ArcFace consistently produces more well-separated identities than ElasticFace. Among the training datasets, FFHQ and WebFace4M achieve comparable scalability and substantially outperform CASIA-WebFace..

\mathcal{Z}=\{z_{i}=f(x_{i})\}_{i=1}^{N}

Algorithm 1 Training IDPrior

z_{i}\sim p_{\theta}(z)

Algorithm 2 Sampling Synthetic Face Dataset with IDPrior

Figure 6: Identity uniqueness under different training configurations. Percentage of unique identities as the cosine-similarity threshold varies from 0 to 1. We compare IDPrior trained using ArcFace and ElasticFace embeddings, as well as IDPrior trained on WebFace4M, CASIA-WebFace, and FFHQ. ArcFace consistently achieves slightly higher identity uniqueness than ElasticFace, while FFHQ and WebFace4M produce comparable results and substantially outperform CASIA-WebFace.

Panel of Figure 8 — Figure 8: Sample images of novel identities generated by different DM models and using different identity generation approaches. Images are generated from different reference identities.

Panel of Figure 9 — Figure 9: Effect of the sampling strategy on scalability. Number of well-separated identities as the number of generated identities increases for DDPM (1000 steps) and DDIM with 25, 50, and 100 denoising steps. DDIM closely matches the scalability of DDPM while requiring substantially fewer denoising steps. All results are reported using IDPrior trained on embeddings extracted from the ArcFace model of the WebFace4M dataset.

Figure 10: Effect of the sampling strategy on identity uniqueness. Percentage of unique identities as the cosine-similarity threshold varies from 0 to 1 for DDPM (1000 steps) and DDIM with 25, 50, and 100 denoising steps. All sampling strategies produce nearly identical identity uniqueness across the full range of thresholds. All results are reported using IDPrior trained on embeddings extracted from the ArcFace model of the WebFace4M dataset.

Panel of Figure 11 — Figure 11: Scalability evaluation, including unconditional IP-Adapter-FaceID. The number of well-separated identities vs. the total number of generated identities increases. Unconditional sampling from IP-Adapter-FaceID quickly saturates, producing fewer than 3% well-separated identities due to severe identity collisions. These results justify the use of a PCA-based baseline for IP-Adapter-FaceID in the main paper.

Figure 12: Identity uniqueness including unconditional IP-Adapter-FaceID. Percentage of unique identities as the cosine-similarity threshold varies from 0 to 1. Unconditional sampling from IP-Adapter-FaceID produces fewer than 3% unique identities across most thresholds, substantially underperforming other approaches.

method	samples	[email protected]	[email protected]	[email protected]	method	samples	0.25	0.30	0.40
Arc2Face/Vec2Face (PCA)	1000	97.10	99.90	100.00	DCFace	1000	44.40	74.50	94.30
	2500	92.96	99.80	100.00		2500	25.16	58.08	90.84
	5000	85.78	99.62	100.00		5000	14.84	44.84	87.16
	7500	79.83	99.39	100.00		7500	10.57	37.56	85.41
	10000	74.35	99.15	100.00		10000	8.11	32.10	83.75
	15000	64.83	98.85	100.00		15000	5.47	25.19	81.09

Table 7: Scalability of identity generation from 1K to 1M identities, which extends the results of the main submission. We report the percentage of unique identities at thresholds 0.25, 0.30, and 0.40. IDPrior consistently maintains the highest uniqueness across all scales, substantially outperforming image-generation-based and PCA-based approaches. We additionally reported results of authentic datasets, CASIA-WebFace (10K identities) and WebFace4M (206K identities). Identities of unconditional IP-Adapter-FaceID sampling saturated at 10K identities. Given the page size, the other models’ results are provided in Table 8 .

method	Samples	[email protected]	[email protected]	[email protected]
IDPrior (Flow-model)	1000	92.30	99.60	100.00
	2500	83.56	98.88	100.00
	5000	70.84	98.10	99.98
	7500	61.25	97.68	99.99
	10000	54.05	97.04	99.99
	15000	43.20	96.04	99.99

Table 8: Scalability of identity generation from 1K to 1M identities, which extends the results of the main submission. This table is complementary to Table 7 .

Figure 13: Histogram of the maximum cosine similarity between each generated identity embedding and all other generated identities within each dataset. Lower similarity indicates better inter-class separation and fewer identity collisions. IDPrior shifts the distribution toward lower similarities compared with image-generation-based and PCA-based approaches, demonstrating its ability to generate more distinct identities.

Synthetic Face Generation

See all 8 papers

IDperturb: Enhancing Variation in Synthetic Face Generation via Angular Perturbation
Feb 21, 2026Fadi Boutros, Eduarda Caldeira, Tahar Chettaoui +1Synthetic Face Generation Synthetic Data Generation
Fraunhofer Institute for Computer Graphics Research IGD, Germany · TU Darmstadt, Germany
Identity-Conditioned Latent Consistency Distillation for Face Synthesis
Aug 31, 2026Tiago Kienen Chaves, Bernardo Biesseck, David MenottiSynthetic Data Augmentation Latent Diffusion Models
Department of Informatics, Federal University of Paraná, Curitiba, Brazil · Federal Institute of Mato Grosso (IFMT), Pontes e Lacerda, MT, Brazil
SynthFaces: Balanced Large Scale Human Dataset
Oct 29, 2026Sergey Karpukhin, Denis Bobkov, Kirill Morozov +1Synthetic Face Generation Synthetic Data Generation
FusionBrain Lab, AXXX Moscow, Russia · HSE University Moscow, Russia · MSU Moscow, Russia

Feb 21, 2026J/K move · Enter open · S save

IDperturb: Enhancing Variation in Synthetic Face Generation via Angular Perturbation

Fadi Boutros, Eduarda Caldeira, Tahar Chettaoui, Naser Damer

Fraunhofer Institute for Computer Graphics Research IGD, Germany · TU Darmstadt, Germany

Synthetic Face Generation Synthetic Data Generation Face Recognition Image Generation Identity-Preserving Image Generation

Open paper AskPDF arXiv

Synthetic data has emerged as a practical alternative to authentic face datasets for training face recognition (FR) systems, especially as privacy and legal concerns increasingly restrict the use of real biometric data. Recent advances in identity-conditional diffusion models have enabled the generation of photorealistic and identity-consistent face images. However, many of these models suffer from limited intra-class variation, an essential property for training robust and generalizable FR models. In this work, we propose IDPERTURB, a simple yet effective geometric-driven sampling strategy to enhance diversity in synthetic face generation. IDPERTURB perturbs identity embeddings within a constrained angular region of the unit hyper-sphere, producing a diverse set of embeddings without modifying the underlying generative model. Each perturbed embedding serves as a conditioning vector for a pre-trained diffusion model, enabling the synthesis of visually varied yet identity-coherent face images suitable for training generalizable FR systems. Empirical results demonstrate that training FR on datasets generated using IDPERTURB yields improved performance across multiple FR benchmarks, compared to existing synthetic data generation approaches.

Abstract

Synthetic Face Generation

Explore similar work

IDperturb: Enhancing Variation in Synthetic Face Generation via Angular Perturbation

Identity-Conditioned Latent Consistency Distillation for Face Synthesis

SynthFaces: Balanced Large Scale Human Dataset