Organizations: School of Cyberspace Security, Hangzhou Dianzi University, Hangzhou, China · Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China · School of Software Engineering, Xi’an Jiaotong University, Xi’an, China · Lishui University and Lishui Institute of Hangzhou Dianzi University, Lishui, China · School of Faculty of Applied Science, Macao Polytechnic University, Macao, China · School of Communication Engineering, Hangzhou Dianzi University, Hangzhou, China
Training generative models with limited data presents severe challenges of mode collapse. A common approach is to adapt a large pretrained generative model upon a target domain with very few samples (fewer than 10), known as few-shot generative model adaptation. However, existing methods often suffer from forgetting source domain identity knowledge during adaptation, which degrades the quality of generated images in the target domain. To address this, we propose Identity Injection and Preservation (I2P), which leverages identity injection and consistency alignment to preserve the source identity knowledge. Specifically, we first introduce an identity injection module that integrates source domain identity knowledge into the target domain's latent space, ensuring the generated images retain key identity knowledge of the source domain. Second, we design an identity substitution module, which includes a style-content decoupler and a reconstruction modulator, to further enhance source domain identity preservation. We enforce identity consistency constraints by aligning features from identity substitution, thereby preserving identity knowledge. Both quantitative and qualitative experiments show that our method achieves substantial improvements over state-of-the-art methods on multiple public datasets and 5 metrics.
Figures & tables
Fig. 1: Few-shot generative model. Given a source generative model GS trained on a large-scale training dataset (such as FFHQ), and adapt to the target domain to get the target generative model GT by using extremely few (such as 10) training datasets
Fig. 2: The framework of our method. a) Identity injection obtains and fuses the latent features of the mapping networks in the source and target domains, thereby guiding the target domain mapping network to remain the source domain identity knowledge. b) Identity substitution module includes a style-content decoupler that decomposes style information features and identity knowledge features, and a reconstruction modulator for reconstructing features using style information features and identity knowledge features. c) Identity consistency consist of style constraint Ls , content constraint Lc , and synthesis constraint Lr . d) Internal structure of style-content decoupler. e) Internals of the reconstruction modulator.
Fig. 3: Cross-domain adaptation performance comparison on Flickr-Faces-HQ (FFHQ) → Sketches task. (a) Baseline methods (TGAN, FreezeD, MineGAN) exhibit overfitting artifacts in 10-shot setting; (b) Recent approaches (CDC, RSSA, PIR) show improved alignment but suffer from content distortion or style inconsistency; (c) Our I 2 P framework maintains structural fidelity and stylistic coherence through identity-preserved adaptation. Visualizations demonstrate our method’s capability to generate high-fidelity sketches while preserving source domain identity features from identical latent codes, significantly outperforming state-of-the-art methods in both quality and cross-domain consistency.
Fig. 4: The cross-domain adaptation results of our I 2 P are shown in the conversion scenarios between FFHQ and four different target domains: FFHQ → Sketches, FFHQ → Sunglasses, FFHQ → Metfaces and FFHQ → Babies, Qualitative results for each target domain illustrate the ability of our I 2 P to adapt the same source domain model to different target domains while preserving domain-invariant features.
Fig. 5: The cross-domain adaptation results of our I 2 P are shown in four different conversion scenarios: (a) LSUN-Churches → Haunted-Houses/VanGogh(top left); (b) LSUN-Cars → Haunted-Cars/Landscape-Cars(top right); (c) AFHQ-Cat → VanGogh(bottom left); (d) AFHQ-Dog → Impressionism(bottom right). Each quadrant illustrates our I 2 P’s capability in preserving domain-invariant features while adapting to diverse artistic styles.
Fig. 6: The 5-shot generation results validate the capability of our I 2 P in cross-domain adaptation on fewer samples, achieving faithful style transfer while maintaining identity knowledge with the source domain.
FID ↓
Intra-LPIPS ↑
Method
babies
metfaces
sketches
sunglasses
babies
metfaces
sketches
sunglasses
TGAN(2018)
104.79
76.81
53.42
55.61
−
−
−
−
FreezeD(2020)
113.01
79.33
50.53
51.29
−
−
−
−
MineGAN(2020)
100.15
84.27
69.77
68.91
−
−
−
−
CDC(2021)
74.39
80.00
45.67
42.13
0.450
0.440
0.605
0.312
RSSA(2022)
75.67
77.06
54.58
44.35
0.605
0.488
0.635
0.400
TABLE I: Quantitative evaluation of methods by FID score( ↓ ) and Intra-LPIPS distance( ↑ ).The source domain comes from FFHQ, and the target domain dataset comes from FFHQ-Babies, Metafaces, sketches, and FFHQ-Sunglasses.
DINO ↑
CLIP-I ↑
CLIP-T ↑
Method
b
m
sk
su
b
m
sk
su
b
m
sk
su
CDC(2021)
0.620
0.601
0.560
0.744
0.601
0.548
0.556
0.640
0 .166
0.197
0.203
0.186
RSSA(2022)
0 .630
0 .602
0 .580
0.788
0 .602
0.555
0 .601
0.675
0.166
0.195
0.205
0.192
PIR(2024)
0.621
0.592
0.568
0.752
0.588
0 .557
0.567
0.642
0.162
0 .199
0 .205
0.189
I 2 P(Ours)
0.639
0.618
0.600
0 .771
0.603
0.563
0.618
0 .667
0.167
0.202
0.206
0 .192
TABLE II: Quantitative evaluation of methods by DINO score( ↑ ), CLIP-I score( ↑ ) and CLIP-T score( ↑ ).The source domain comes from FFHQ, and the target domain dataset comes from FFHQ-Babies, Metafaces, sketches, and FFHQ-Sunglasses.
IP
FID ↓
II
Lr
Lc & Ls
b
m
sk
su
✗
✗
✗
79.99
77.15
44.55
45.05
✗
✓
✓
77.54
75.33
39.36
43.55
✓
✗
✗
74.63
74.05
39.09
39.40
✓
✗
✓
74.27
73.09
38.52
38.43
✓
✓
✗
72.67
69.35
38.20
38.21
TABLE III: Quantitative ablation experiments on FID ↓ of our I 2 P method. II refers to the identity injection module. IP refers to the identity preservation, including identity substitution and identity consistency.
Fig. 7: Qualitative ablation of the I 2 P’s modules. II refers to the identity injection module. IP refers to the identity preservation, including identity substitution and identity consistency.
IP
Intra-LPIPS ↑
II
Lr
Lc&Ls
b
m
sk
su
✗
✗
✗
0.449
0.396
0.486
0.349
✗
✓
✓
0.551
0.482
0.592
0.364
✓
✗
✗
0.542
0.463
0.579
0.363
✓
✗
✓
0.601
0.493
0.622
0.382
✓
✓
✗
0.583
0.507
0.612
0.397
TABLE IV: Quantitative experiments on Intra-LPIPS ↑ of our I 2 P method. II refers to the identity injection module. IP refers to the identity preservation, including identity substitution and identity consistency.
Fig. 8: Qualitative ablation experiments of our I 2 P on two types of losses in the Identity Preservation module: synthesis constraint and content constraint & style constraint
Fig. 9: Visualization experiment of synthesis feature MCSST . We use the style-content features of the source domain and the target domain to construct the synthesis feature, and then train an image decoder with the corresponding original image as the training set, and then input the synthesis feature MCSST to obtain the visualization image.
Fig. 10: Qualitative ablation of the coefficient α in Eq. 2 , which means the degree of identity injection
Fig. 11: Qualitative and quantitative ablation experiments on the proportion of two types of losses in the Identity Preservation module: synthesis constraint and content constraint & style constraint.
Fig. 12: Qualitative ablation of the coefficient λ in Eq. 8 , which means the weights of the identity consistency
CDC
RSSA
PIR
I 2 P
Time(min)
80
103
130
88
Model size(MB)
115.28
115.28
115.28
115.28
Memory size(GB)
12.9
18.9
19.2
14.7
TABLE V: Computational complexity experiment with baseline
Yeqi He received the B.E. degree in intelligence science and technology from Hangzhou Dianzi University, Hangzhou, China, in 2023. He is currently pursuing the Ph.D. degree with Hangzhou Dianzi University, Hangzhou, China. His research interests include computer vision and generative models.
Table 18
Liang Li received his B.S. degree from Xi’an Jiaotong University in 2008, and Ph.D. degree from Institute of Computing Technology, Chinese Academy of Sciences, Beijing, China in 2013. From 2013 to 2015, he held a post-doc position with the Department of Computer and Control Engineering, University of Chinese Academy of Sciences, Beijing, China. Currently he is serving as the associate professor at Institute of Computing Technology, Chinese Academy of Sciences. He has also served on a number of committees of international journals and conferences. Dr. Li has published over 100 refereed journal/conference papers, including IEEE TPAMI, IJCV, TNNLS, TIP, CVPR, ICCV. His research interests include multimedia content analysis, computer vision, and pattern recognition.
Table 19
Jiehua Zhang received the B.E. degree from Hunan Institute of Engineering, Xiangtan, China, in 2019, and M.S. degree from Hangzhou Dianzi University, Hangzhou, China, in 2022. He is currently pursuing the Ph.D. degree with Xi’an Jiaotong University, Xi’an, China. His main research interests include computer vision and transfer learning.
Table 20
Yaoqi Sun received the BS degree from Hangzhou Dianzi University, Hangzhou, China, in 2016. He is currently working toward the PHD degree at Hangzhou Dianzi University. His major research interest includes computer vision problems related to transfer learning. He has published over 20 refereed journal/conference papers, including IEEE TPAMI, TCSVT, ACM TOMM, ACM MM, etc.
Table 21
Xichun Sheng received the M.S. degree in Materials Science and Engineering from Xiangtan University. Currently pursuing the Ph.D. degree in Computer Technology at Macau Polytechnic University. Researching interests include visual language models, which integrate computer vision and natural language processing to enhance the capabilities of artificial intelligence systems in understanding and generating multimodal content.
Table 22
Zhidong Zhao (Member, IEEE) received the B.S. and M.S. degrees in mechanical engineering from the Nanjing University of Science and Technology, Nanjing, China, in 1998 and 2001, respectively, and the Ph.D. degree in biomedical engineering from Zhejiang University, Hangzhou, China, in 2004. He is currently a Full Professor with Hangzhou Dianzi University, Hangzhou. His research interests include biomedical signal processing, wireless sensor networks, biometrics, and machine learning.
Table 23
Chenggang Yan received the B.S. degree in control science and engineering from Shandong University, Jinan, China, in 2008, and the Ph.D. degree in computer science from the Chinese Academy of Sciences University, Beijing, China, in 2013. He is currently a Professor with the Department of Automation, Hangzhou Dianzi University, Hangzhou, China. His research interests include computational photography, pattern recognition, and intelligent systems.
Adaptation of blackbox generative models has been widely studied recently through the exploration of several methods including generator fine-tuning, latent space searches, leveraging singular value decomposition, and so on. However, adapting large-scale generative AI tools to specific use cases continues to be challenging, as many of these industry-grade models are not made widely available. The traditional approach of fine-tuning certain layers of a generative network is not feasible due to the expense of storing and fine-tuning generative models, as well as the restricted access to weights and gradients. Recognizing these challenges, we propose a novel end-to-end pipeline aimed at domain adaptation by leveraging geometry-preserving loss functions in conjunction to pre-trained generative adversarial networks (GANs). Our method rethinks the problem of adaptation by re-contextualizing the role of GAN inversion in obtaining accurate latent space representations. Extending the ability of existing state-of-the-art inverters, we preserve pair-wise distances between tangent spaces to successfully train a latent generative model to produce samples from the target distribution. We evaluate our proposed pipeline on StyleGANs with real distribution shifts and demonstrate that the introduction of the geometry preserving loss function lends to improved adaptation of generative models compared to other traditional loss functions.
Identity-preserved image generation is typically built on many-step diffusion backbones, making personalized generation expensive at deployment time. We show that this cost is often unnecessary for identity-conditioned FLUX generation. A frozen InfuseNet identity adapter trained with dev transfers directly to the distilled schnell backbone without retraining. This two-line replacement -- changing the backbone path and disabling classifier-free guidance -- reduces latency by 5.9x while improving ArcFace identity similarity by +0.028 and lpips by -0.016 over the standard 28-step dev baseline. To explain why this works, we analyze the denoising trajectory and find that identity fidelity enters an early effective regime, often within 4-8 steps, while later steps primarily refine visual detail, sharpness, and contrast. Adapter ablations confirm that identity formation depends on the identity adapter, while attention-stream norm probes suggest that the relative conditioning contribution decreases as sampling proceeds. Preliminary style-adapter and object-adapter sweeps on SDXL and SD1.5 show similar diminishing returns after intermediate steps. These results position distilled backbone replacement as a simple, training-free strategy for improving the efficiency-fidelity tradeoff of identity-preserved generation.
Generative image models can now produce high-quality images, follow complex instructions, and support precise edits, but they still struggle to preserve who or what is being depicted. When generating or editing images of a specific subject, identity may drift as the pose, expression, appearance, viewpoint, or surrounding scene changes. Existing subject-driven methods make fundamentally different choices about where identity is represented: through the input context (GPT-Image-2, NB2), as trainable subject-specific model parameters (LoRA), or as a persistent identity layer (PHOTA IDENTITY) reusable across generations and edits. We systematically benchmark these paradigms across subject-driven generation, editing, restoration, and multi-subject settings, with tasks designed to increasingly stress identity preservation. Our results show that identity preservation remains a distinct limitation of current generative foundation models: strong image quality and instruction following do not necessarily imply strong identity fidelity, and identity degradation becomes more pronounced under iterative edits, small subject scales, severe image degradation, and multi-subject composition. Persistent identity substantially reduces this degradation across generation, editing, and restoration, consistently improving identity preservation when applied to different foundation models while maintaining comparable instruction adherence and perceptual image quality. These results suggest that identity does not simply emerge from increasingly capable generative models, but can instead be represented as persistent subject knowledge that is composed independently with the underlying generative model.