UniBioTransfer: A Unified Framework for Multiple Biometrics Transfer

Caiyi Sun; Yujing Sun; Xiangyu Li; Yuhang Zheng; Yiming Ren; Jiamin Wang; Yuexin Ma; Siu-Ming Yiu

cs.CVMar 20, 2026

UniBioTransfer: A Unified Framework for Multiple Biometrics Transfer

Authors: Caiyi Sun, Yujing Sun, Xiangyu Li, Yuhang Zheng, Yiming Ren, Jiamin Wang, Yuexin Ma, Siu-Ming Yiu

Organizations: The University of Hong Kong · Digital Trust Centre, Nanyang Technological University · ShanghaiTech University

Abstract

Deepface generation has traditionally followed a task-driven paradigm, where distinct tasks (e.g., face transfer and hair transfer) are addressed by task-specific models. Nevertheless, this single-task setting severely limits model generalization and scalability. A unified model capable of solving multiple deepface generation tasks in a single pass represents a promising and practical direction, yet remains challenging due to data scarcity and cross-task conflicts arising from heterogeneous attribute transformations. To this end, we propose UniBioTransfer, the first unified framework capable of handling both conventional deepface tasks (e.g., face transfer and face reenactment) and shape-varying transformations (e.g., hair transfer and head transfer). Besides, UniBioTransfer naturally generalizes to unseen tasks, like lip, eye, and glasses transfer, with minimal fine-tuning. Generally, UniBioTransfer addresses data insufficiency in multi-task generation through a unified data construction strategy, including a swapping-based corruption mechanism designed for spatially dynamic attributes like hair. It further mitigates cross-task interference via an innovative BioMoE, a mixture-of-experts based model coupled with a novel two-stage training strategy that effectively disentangles task-specific knowledge. Extensive experiments demonstrate the effectiveness, generalization, and scalability of UniBioTransfer, outperforming both existing unified models and task-specific methods across a wide range of deepface generation tasks. Project page is at https://scy639.github.io/UniBioTransfer.github.io/

Figures & tables

Figure 3: Our unified data corruption strategy for different attribute types. (a) Relative-static attributes: the target is constructed by simple masking or data augmentation of the GT image. (b) Spatially-dynamic attributes: we utilize our swapping-based corruption strategy, which employs an off-the-shelf generative model to replace specific attributes in the GT with arbitrary novel variations, preventing shape leakage from mask boundaries.

Figure 4: UniBioTransfer architecture overview. (a) Overall framework. (b) We introduce an MoE-enhanced Feed Forward Network (FFN). (c) Expert selection is guided by a Structure-Aware Router. (d) The entire system is optimized using a two-stage training strategy designed to stabilize routing and promote expert specialization.

Figure 5: Visual Comparisons on diverse deepface tasks. More results in Suppl.

Method	Medium Level Tasks									High Level Tasks
	Face Transfer			Hair Transfer			Motion Transfer			Head Transfer
	ID sim $\uparrow$	pose dist. $\downarrow$	expr. dist. $\downarrow$	CLIP dist. $\downarrow$	ID sim $\uparrow$	non-hair SSIM $\uparrow$	ID sim $\uparrow$	pose dist. $\downarrow$	expr. dist. $\downarrow$	CLIP dist. $\downarrow$	ID sim $\uparrow$	pose dist. $\downarrow$
Multi-Task Methods (Training separately for different tasks)
REFace (WACV 25)	0.631	3.75	1.04	-	-	-	-	-	-	0.639	0.540	5.43
Unified Method (Train once for all tasks)
Face-Adapter (ECCV 24)	0.519	4.91	1.33	-	-	-	0.503	6.89	2.27	-	-	-

Table 1: Quantitative comparisons on High-Level and Mid-Level deepface tasks with multi-task and unified approaches . Notably, only UniBioTransfer can perform all four tasks simultaneously.For all metrics,

\uparrow

indicates higher is better and

\downarrow

indicates lower is better. Best results are highlighted in bold .

Method	Medium Level Tasks									High Level Tasks
	Face Transfer			Hair Transfer			Motion Transfer			Head Transfer
	ID sim $\uparrow$	pose dist. $\downarrow$	expr. dist. $\downarrow$	CLIP dist. $\downarrow$	ID sim $\uparrow$	non-hair SSIM $\uparrow$	ID sim $\uparrow$	pose dist. $\downarrow$	expr. dist. $\downarrow$	CLIP dist. $\downarrow$	ID sim $\uparrow$	pose dist. $\downarrow$
HunyuanPortrait (CVPR 25)	-	-	-	-	-	-	0.580	9.22	2.18	-	-	-
CanonSwap (ICCV 25)	0.551	2.92	0.87	-	-	-	-	-	-	-	-	-
Stable-Hair (AAAI 25)	-	-	-	0.468	0.855	0.87	-	-	-	-	-	-
HairFusion (AAAI 25)	-	-	-	0.478	0.848	0.80	-	-	-	-	-	-

Table 2: Quantitative comparisons on High-Level and Mid-Level deepface tasks with task-specific methods . Notably, UniBioTransfer performs all the four tasks simultaneously in a single training cycle.

Method	Facial Part Transfer (Low Level Tasks)
Method	Eye			Nose			Lip
	CLIP L2 $\downarrow$	expr L2 $\downarrow$	SSIM $\uparrow$	CLIP L2 $\downarrow$	expr L2 $\downarrow$	SSIM $\uparrow$	CLIP L2 $\downarrow$	expr L2 $\downarrow$	SSIM $\uparrow$
FuseAnyPart (NIPS 24)	0.449	0.798	0.898	0.332	0.802	0.898	0.372	0.794	0.902
Ours (w/o Uni.)	0.471	0.697	0.889	0.337	0.672	0.859	0.384	1.009	0.867
Ours	0.427	0.599	0.927	0.327	0.532	0.937	0.378	0.579	0.937

Table 3: Quantitative comparisons on Low-Level Facial-Part Swap tasks.

Appendix figures & tables18 assets

Supplementary material from the paper’s appendix.

Appendix

Figure 7: We show the specific examples of data construction on 4 tasks in (a)-(d). Grey arrow means the corruption process only involves Relative-Static Attributes Corruption. Yellow arrow means it involves Spatially-Dynamic Attributes Corruption, thus need the Swapping-Based Corruption, as shown in (e).

Method	Face transfer $\uparrow$	Hair transfer $\downarrow$	Reenact $\downarrow$	Head transfer $\downarrow$
Qwen-Image-Edit (arXiv 25)	0.442	0.486	17.13	0.582
Ours	0.637	0.421	7.09	0.460

Appendix

Table 6: Comparison with Qwen-Image-Edit (general-purpose image editing model). For each task, we report the most representative metric, that is identity similarity, hair clip distance, pose distance, and head clip distance. For all metrics,

\uparrow

indicates higher is better and

\downarrow

indicates lower is better. Best results are in bold .

Method	CLIP dist. $\downarrow$	ID sim $\uparrow$	non-hair SSIM $\uparrow$
Stable-Hair (AAAI 25)	0.468	0.855	0.87
Our initial model	0.421	0.887	0.91
Our evolved model	0.419	0.888	0.94

Appendix

Table 7: Quantitative results of the self-evolution experiment on hair transfer task. By leveraging our initial model to construct higher-quality training data, we can obtain an evolved model with improved performance.

Figure 8: Analysis of the general image editing model Qwen-Image-Edit [ 60 ] . (a) Under the multiple-image mode (reference-based editing), Qwen-Image-Edit exhibits many unstable behavior like under-follow the reference, over-transfer and damage the target, or hallucinate in the edited region. (More examples are shown in Section 0.D : Fig. 11 - 14 ). The consistently poor quantitative results shown in Tab. 1 further confirm its unsuitability for the transfer task . (b) However, under the single-image mode (instruction-based editing), it can still generate plausible and diverse attribute variations, which is sufficient for swapping-based corruption .

LoRA Rank Allocation Method	Face transfer $\uparrow$	Hair transfer $\downarrow$	Reenact $\downarrow$	Head transfer $\downarrow$
Uniform	0.636	0.419	10.13	0.460
Gradient-based	0.634	0.433	7.35	0.464
Result-based ( Ours )	0.637	0.421	7.09	0.460

Appendix

Table 8: We compare our Result-based Adaptive LoRA rank against the conventional gradient-based and the uniform rank. For each task, we report the most representative metric, that is identity similarity, hair clip distance, pose distance, and head clip distance.

Figure 9: Examples of discarded and retained image pairs after filtering for hair transfer. (a) Discarded pairs due to low background SSIM, low face identity similarity, large pose L2 distance, and large expression L2 distance, respectively; (b) Retained pairs after filtering.

Figure 10: Qualitative comparison against the StyleGAN-based method, Barbershop [ 79 ] . As indicated by the red arrows, Barbershop encounters obvious visual artifacts and distortions when there is a pose discrepancy between the reference and target images. Moreover, due to its heavy reliance on latent optimization, it requires $\geq 4$ minutes to infer a single image pair, rendering it highly unscalable and unsuitable for our unified, efficient multi-task generation task.

Method	CLIP dist. $\downarrow$	ID sim $\uparrow$	pose dist. $\downarrow$	expr. dist. $\downarrow$	CLIP dist. (hair) $\downarrow$	FID $\downarrow$
Task-Specific Methods
GHOST2.0 (arXiv 25)	0.618	0.461	5.78	1.867	0.497	31.38
Multi-Task Methods (training separately for different tasks)
REFace (WACV 25)	0.639	0.540	5.43	1.776	0.549	8.33
Unified Methods (train once for all tasks)
Ours	0.460	0.545	5.28	1.772	0.399	8.28

Appendix

Table 9: Quantitative comparison with more metrics for head transfer. For all metrics,

\uparrow

indicates higher is better and

\downarrow

indicates lower is better. Best result from Unified Methods are marked Red . Overall best results are highlighted in bold , while overall second-best results are underlined .

Method	Peak CUDA memory (GB) $\downarrow$	Infer time (second/img) $\downarrow$
Task-Specific Methods
CanonSwap (ICCV 25)	11.7	$<1$
Stable-Hair (AAAI 25)	19.3	16
HairFusion (AAAI 25)	10.9	22
HunyuanPortrait (CVPR 25)	18.4	103
GHOST2.0 (arXiv 25)	8.0	$<1$

Appendix

Table 10: Efficiency comparison: Runtime and Peak CUDA memory. For all metrics,

\downarrow

indicates lower is better. For multi-task methods, we report the maximum runtime and peak CUDA memory across all tasks.

Task	Method	Exaggerated Expressions			Extreme Pose			Occlusions
		ID sim $\uparrow$ pose dist. $\downarrow$ expr. dist. $\downarrow$			ID sim $\uparrow$ pose dist. $\downarrow$ expr. dist. $\downarrow$			ID sim $\uparrow$ pose dist. $\downarrow$ expr. dist. $\downarrow$
Face Transfer	REFace	0.509	5.10	1.28	0.293	10.09	1.10	0.311	5.93	1.07
	CanonSwap	0.446	2.27	0.91	0.141	3.73	0.75	0.152	2.53	0.76
	Ours	0.518	3.54	1.04	0.302	4.25	0.73	0.316	4.80	1.06
		CLIP dist. $\downarrow$	ID sim $\uparrow$	non-hair SSIM $\uparrow$	CLIP dist. $\downarrow$	ID sim $\uparrow$	non-hair SSIM $\uparrow$	CLIP dist. $\downarrow$	ID sim $\uparrow$	non-hair SSIM $\uparrow$
Hair Transfer	Stable-Hair	0.460	0.869	0.871	0.477	0.831	0.880	0.472	0.826	0.848

Appendix

Table 11: Comparisons on the most competitive methods for each task under complex scenes.

Figure 11: Hair transfer visual comparison.

Figure 12: Motion transfer visual comparison. HunyuanPortrait tends to change the face skin, or lighting. The faceAdapter shows suboptimal face fidelity.

Figure 13: Head transfer visual comparison.

Figure 14: Face transfer visual comparison.

Figure 15: Visual comparison of face transfer under complex scenes.

Figure 16: Visual comparison of hair transfer under complex scenes.

Figure 17: Visual comparison of motion transfer (face reenactment) under complex scenes.

Figure 18: Visual comparison of head transfer under complex scenes.

Image Editing

Image editing changes selected visual content while preserving the parts of an image that should remain unchanged.

See all 315 papers

Transferability Between Understanding and Generation in Unified Multimodal Models
Jul 5, 2026Jiwon Kang, Heeji Yoon, Jaewoo Jung +5Unified Multimodal Models Multimodal Generation
KAIST AI, South Korea · Blynx, South Korea · Trillionlabs, South Korea
MMFace-DiT: A Dual-Stream Diffusion Transformer for High-Fidelity Multimodal Face Generation
Mar 30, 2026Bharath Krishnamurthy, Ajita RattaniMultimodal Generation Controllable Image Generation
University of North Texas, Denton, TX, USA
OrthoTryOn: Geometric Orthogonalization for Conflict-Free Unified Fashion Generation
Jun 26, 2026Zhaotong Yang, Ying Tai, Jiahui Zhan +3Diffusion Model Guidance Virtual Try-On
PCA Lab, School of Computer Science and Engineering, Nanjing University of Science and Technology · PCA Lab, School of Intelligence Science and Technology, Nanjing University · Shanghai Jiao Tong University

Jul 5, 2026J/K move · Enter open · S save

Transferability Between Understanding and Generation in Unified Multimodal Models

Jiwon Kang, Heeji Yoon, Jaewoo Jung, Jaewon Min+4

KAIST AI, South Korea · Blynx, South Korea · Trillionlabs, South Korea

Unified Multimodal Models Multimodal Generation Transfer Learning Multimodal Understanding

Open paper AskPDF arXiv

Unified Multimodal Models (UMMs) integrate image understanding and generation within a single architecture, yet how the two tasks interact remains understudied. We investigate

\boldsymbol{\mathsf{transferability}}

in UMMs: whether training a capability on one task improves the same capability on the other without explicit supervision. Through controlled experiments, we empirically find that transferability depends on architecture-models with fully shared transformer backbone and a unified visual encoder exhibit consistent cross-task transfer, while loosely coupled designs show little or none. Leveraging this transferability, we propose a practical training strategy. The most straightforward way to improve a target generative capability (e.g., counting) is to fine-tune generation directly, but this can degrade visual quality due to distribution shift. Instead, we train the corresponding understanding task and let it transfer into generation, which improves capability-specific generative performance while minimizing distribution shift. We validate this across three capabilities-counting, spatial relation, and text recognition/generation-showing that cross-task transferability can be systematically exploited in UMMs.

Abstract

Image Editing

Explore similar work

Transferability Between Understanding and Generation in Unified Multimodal Models

MMFace-DiT: A Dual-Stream Diffusion Transformer for High-Fidelity Multimodal Face Generation

OrthoTryOn: Geometric Orthogonalization for Conflict-Free Unified Fashion Generation