Few-Shot Generative Model Adaption via Identity Injection and Preservation

Yeqi He; Liang Li; Jiehua Zhang; Yaoqi Sun; Xichun Sheng; Zhidong Zhao; Chenggang Yan

cs.CVMar 24, 2026

Few-Shot Generative Model Adaption via Identity Injection and Preservation

Authors: Yeqi He, Liang Li, Jiehua Zhang, Yaoqi Sun, Xichun Sheng, Zhidong Zhao, Chenggang Yan

Organizations: School of Cyberspace Security, Hangzhou Dianzi University, Hangzhou, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · School of Software Engineering, Xi’an Jiaotong University, Xi’an, China · Lishui University and Lishui Institute of Hangzhou Dianzi University, Lishui, China · School of Faculty of Applied Science, Macao Polytechnic University, Macao, China · School of Communication Engineering, Hangzhou Dianzi University, Hangzhou, China

Abstract

Training generative models with limited data presents severe challenges of mode collapse. A common approach is to adapt a large pretrained generative model upon a target domain with very few samples (fewer than 10), known as few-shot generative model adaptation. However, existing methods often suffer from forgetting source domain identity knowledge during adaptation, which degrades the quality of generated images in the target domain. To address this, we propose Identity Injection and Preservation (I $^2$ P), which leverages identity injection and consistency alignment to preserve the source identity knowledge. Specifically, we first introduce an identity injection module that integrates source domain identity knowledge into the target domain's latent space, ensuring the generated images retain key identity knowledge of the source domain. Second, we design an identity substitution module, which includes a style-content decoupler and a reconstruction modulator, to further enhance source domain identity preservation. We enforce identity consistency constraints by aligning features from identity substitution, thereby preserving identity knowledge. Both quantitative and qualitative experiments show that our method achieves substantial improvements over state-of-the-art methods on multiple public datasets and 5 metrics.

Figures & tables

Fig. 1: Few-shot generative model. Given a source generative model $G_{S}$ trained on a large-scale training dataset (such as FFHQ), and adapt to the target domain to get the target generative model $G_{T}$ by using extremely few (such as 10) training datasets

Fig. 2: The framework of our method. a) Identity injection obtains and fuses the latent features of the mapping networks in the source and target domains, thereby guiding the target domain mapping network to remain the source domain identity knowledge. b) Identity substitution module includes a style-content decoupler that decomposes style information features and identity knowledge features, and a reconstruction modulator for reconstructing features using style information features and identity knowledge features. c) Identity consistency consist of style constraint $\mathcal{L}_{s}$ , content constraint $\mathcal{L}_{c}$ , and synthesis constraint $\mathcal{L}_{r}$ . d) Internal structure of style-content decoupler. e) Internals of the reconstruction modulator.

Fig. 3: Cross-domain adaptation performance comparison on Flickr-Faces-HQ (FFHQ) $\rightarrow$ Sketches task. (a) Baseline methods (TGAN, FreezeD, MineGAN) exhibit overfitting artifacts in 10-shot setting; (b) Recent approaches (CDC, RSSA, PIR) show improved alignment but suffer from content distortion or style inconsistency; (c) Our I 2 P framework maintains structural fidelity and stylistic coherence through identity-preserved adaptation. Visualizations demonstrate our method’s capability to generate high-fidelity sketches while preserving source domain identity features from identical latent codes, significantly outperforming state-of-the-art methods in both quality and cross-domain consistency.

Fig. 4: The cross-domain adaptation results of our I 2 P are shown in the conversion scenarios between FFHQ and four different target domains: FFHQ $\rightarrow$ Sketches, FFHQ $\rightarrow$ Sunglasses, FFHQ $\rightarrow$ Metfaces and FFHQ $\rightarrow$ Babies, Qualitative results for each target domain illustrate the ability of our I 2 P to adapt the same source domain model to different target domains while preserving domain-invariant features.

Fig. 5: The cross-domain adaptation results of our I 2 P are shown in four different conversion scenarios: (a) LSUN-Churches $\rightarrow$ Haunted-Houses/VanGogh(top left); (b) LSUN-Cars $\rightarrow$ Haunted-Cars/Landscape-Cars(top right); (c) AFHQ-Cat $\rightarrow$ VanGogh(bottom left); (d) AFHQ-Dog $\rightarrow$ Impressionism(bottom right). Each quadrant illustrates our I 2 P’s capability in preserving domain-invariant features while adapting to diverse artistic styles.

Fig. 6: The 5-shot generation results validate the capability of our I 2 P in cross-domain adaptation on fewer samples, achieving faithful style transfer while maintaining identity knowledge with the source domain.

	FID $\downarrow$				Intra-LPIPS $\uparrow$
Method	babies	metfaces	sketches	sunglasses	babies	metfaces	sketches	sunglasses
TGAN(2018)	104.79	76.81	53.42	55.61	$-$	$-$	$-$	$-$
FreezeD(2020)	113.01	79.33	50.53	51.29	$-$	$-$	$-$	$-$
MineGAN(2020)	100.15	84.27	69.77	68.91	$-$	$-$	$-$	$-$
CDC(2021)	74.39	80.00	45.67	42.13	0.450	0.440	0.605	0.312
RSSA(2022)	75.67	77.06	54.58	44.35	0.605	0.488	0.635	0.400

TABLE I: Quantitative evaluation of methods by FID score(

\downarrow

) and Intra-LPIPS distance(

\uparrow

).The source domain comes from FFHQ, and the target domain dataset comes from FFHQ-Babies, Metafaces, sketches, and FFHQ-Sunglasses.

	DINO $\uparrow$				CLIP-I $\uparrow$				CLIP-T $\uparrow$
Method	b	m	sk	su	b	m	sk	su	b	m	sk	su
CDC(2021)	0.620	0.601	0.560	0.744	0.601	0.548	0.556	0.640	0 .166	0.197	0.203	0.186
RSSA(2022)	0 .630	0 .602	0 .580	0.788	0 .602	0.555	0 .601	0.675	0.166	0.195	0.205	0.192
PIR(2024)	0.621	0.592	0.568	0.752	0.588	0 .557	0.567	0.642	0.162	0 .199	0 .205	0.189
I 2 P(Ours)	0.639	0.618	0.600	0 .771	0.603	0.563	0.618	0 .667	0.167	0.202	0.206	0 .192

TABLE II: Quantitative evaluation of methods by DINO score(

\uparrow

), CLIP-I score(

\uparrow

) and CLIP-T score(

\uparrow

).The source domain comes from FFHQ, and the target domain dataset comes from FFHQ-Babies, Metafaces, sketches, and FFHQ-Sunglasses.

II	$L_{r}$	$L_{c}$ & $L_{s}$	b	m	sk	su
	IP		FID $\downarrow$
✗	✗	✗	79.99	77.15	44.55	45.05
✗	✓	✓	77.54	75.33	39.36	43.55
✓	✗	✗	74.63	74.05	39.09	39.40
✓	✗	✓	74.27	73.09	38.52	38.43
✓	✓	✗	72.67	69.35	38.20	38.21

TABLE III: Quantitative ablation experiments on FID

\downarrow

of our I 2 P method. II refers to the identity injection module. IP refers to the identity preservation, including identity substitution and identity consistency.

Fig. 7: Qualitative ablation of the I 2 P’s modules. II refers to the identity injection module. IP refers to the identity preservation, including identity substitution and identity consistency.

II	$L_{r}$	$L_{c}\&L_{s}$	b	m	sk	su
	IP		Intra-LPIPS $\uparrow$
✗	✗	✗	0.449	0.396	0.486	0.349
✗	✓	✓	0.551	0.482	0.592	0.364
✓	✗	✗	0.542	0.463	0.579	0.363
✓	✗	✓	0.601	0.493	0.622	0.382
✓	✓	✗	0.583	0.507	0.612	0.397

TABLE IV: Quantitative experiments on Intra-LPIPS

\uparrow

of our I 2 P method. II refers to the identity injection module. IP refers to the identity preservation, including identity substitution and identity consistency.

Fig. 8: Qualitative ablation experiments of our I 2 P on two types of losses in the Identity Preservation module: synthesis constraint and content constraint & style constraint

Fig. 9: Visualization experiment of synthesis feature $M_{{C_{S}}{S_{T}}}$ . We use the style-content features of the source domain and the target domain to construct the synthesis feature, and then train an image decoder with the corresponding original image as the training set, and then input the synthesis feature $M_{{C_{S}}{S_{T}}}$ to obtain the visualization image.

Fig. 10: Qualitative ablation of the coefficient $\alpha$ in Eq. 2 , which means the degree of identity injection

Fig. 11: Qualitative and quantitative ablation experiments on the proportion of two types of losses in the Identity Preservation module: synthesis constraint and content constraint & style constraint.

Fig. 12: Qualitative ablation of the coefficient $\lambda$ in Eq. 8 , which means the weights of the identity consistency

	CDC	RSSA	PIR	I 2 P
Time(min)	80	103	130	88
Model size(MB)	115.28	115.28	115.28	115.28
Memory size(GB)	12.9	18.9	19.2	14.7

TABLE V: Computational complexity experiment with baseline

Yeqi He received the B.E. degree in intelligence science and technology from Hangzhou Dianzi University, Hangzhou, China, in 2023. He is currently pursuing the Ph.D. degree with Hangzhou Dianzi University, Hangzhou, China. His research interests include computer vision and generative models.

Table 18

Liang Li received his B.S. degree from Xi’an Jiaotong University in 2008, and Ph.D. degree from Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China in 2013. From 2013 to 2015, he held a post-doc position with the Department of Computer and Control Engineering, University of Chinese Academy of Sciences, Beijing, China. Currently he is serving as the associate professor at Institute of Computing Technology, Chinese Academy of Sciences. He has also served on a number of committees of international journals and conferences. Dr. Li has published over 100 refereed journal/conference papers, including IEEE TPAMI, IJCV, TNNLS, TIP, CVPR, ICCV. His research interests include multimedia content analysis, computer vision, and pattern recognition.

Table 19

Jiehua Zhang received the B.E. degree from Hunan Institute of Engineering, Xiangtan, China, in 2019, and M.S. degree from Hangzhou Dianzi University, Hangzhou, China, in 2022. He is currently pursuing the Ph.D. degree with Xi’an Jiaotong University, Xi’an, China. His main research interests include computer vision and transfer learning.

Table 20

Yaoqi Sun received the BS degree from Hangzhou Dianzi University, Hangzhou, China, in 2016. He is currently working toward the PHD degree at Hangzhou Dianzi University. His major research interest includes computer vision problems related to transfer learning. He has published over 20 refereed journal/conference papers, including IEEE TPAMI, TCSVT, ACM TOMM, ACM MM, etc.

Table 21

Xichun Sheng received the M.S. degree in Materials Science and Engineering from Xiangtan University. Currently pursuing the Ph.D. degree in Computer Technology at Macau Polytechnic University. Researching interests include visual language models, which integrate computer vision and natural language processing to enhance the capabilities of artificial intelligence systems in understanding and generating multimodal content.

Table 22

Zhidong Zhao (Member, IEEE) received the B.S. and M.S. degrees in mechanical engineering from the Nanjing University of Science and Technology, Nanjing, China, in 1998 and 2001, respectively, and the Ph.D. degree in biomedical engineering from Zhejiang University, Hangzhou, China, in 2004. He is currently a Full Professor with Hangzhou Dianzi University, Hangzhou. His research interests include biomedical signal processing, wireless sensor networks, biometrics, and machine learning.

Table 23

Chenggang Yan received the B.S. degree in control science and engineering from Shandong University, Jinan, China, in 2008, and the Ph.D. degree in computer science from the Chinese Academy of Sciences University, Beijing, China, in 2013. He is currently a Professor with the Department of Automation, Hangzhou Dianzi University, Hangzhou, China. His research interests include computational photography, pattern recognition, and intelligent systems.

Table 24

Explore similar work

Geometry Preserving Loss Functions Promote Improved Adaptation of Blackbox Generative Model
Apr 26, 2026Sinjini Mitra, Constantine Kyriakakis, Shenyuan Liang +2Domain Adaptation Latent Space Optimization
1Geometric Media Lab, Arizona State University, Tempe 85281, USA. · Department of Statistics, Florida State University, Tallahassee 32306, USA.
When Few Steps Are Enough: Training-Free Acceleration of Identity-Preserved Generation
May 10, 2026Dongqi ZhengIdentity-Preserving Image Generation Few-Step Diffusion Sampling
Persistent Identity Preservation in Generative Image Models: A Benchmark and Evaluation System
Sep 3, 2026Mengwei Ren, Xuaner Zhang, Zhihao XiaSubject-Driven Image Generation Identity-Preserving Image Generation
Phota Labs Research

Apr 26, 2026J/K move · Enter open · S save

Geometry Preserving Loss Functions Promote Improved Adaptation of Blackbox Generative Model

Sinjini Mitra, Constantine Kyriakakis, Shenyuan Liang, Anuj Srivastava+1

1Geometric Media Lab, Arizona State University, Tempe 85281, USA. · Department of Statistics, Florida State University, Tallahassee 32306, USA.

Domain Adaptation Latent Space Optimization Generative Adversarial Networks

Open paper AskPDF arXiv

Adaptation of blackbox generative models has been widely studied recently through the exploration of several methods including generator fine-tuning, latent space searches, leveraging singular value decomposition, and so on. However, adapting large-scale generative AI tools to specific use cases continues to be challenging, as many of these industry-grade models are not made widely available. The traditional approach of fine-tuning certain layers of a generative network is not feasible due to the expense of storing and fine-tuning generative models, as well as the restricted access to weights and gradients. Recognizing these challenges, we propose a novel end-to-end pipeline aimed at domain adaptation by leveraging geometry-preserving loss functions in conjunction to pre-trained generative adversarial networks (GANs). Our method rethinks the problem of adaptation by re-contextualizing the role of GAN inversion in obtaining accurate latent space representations. Extending the ability of existing state-of-the-art inverters, we preserve pair-wise distances between tangent spaces to successfully train a latent generative model to produce samples from the target distribution. We evaluate our proposed pipeline on StyleGANs with real distribution shifts and demonstrate that the introduction of the geometry preserving loss function lends to improved adaptation of generative models compared to other traditional loss functions.

Abstract

Explore similar work

Geometry Preserving Loss Functions Promote Improved Adaptation of Blackbox Generative Model

When Few Steps Are Enough: Training-Free Acceleration of Identity-Preserved Generation

Persistent Identity Preservation in Generative Image Models: A Benchmark and Evaluation System