U-Face: An Efficient and Generalizable Framework for Unsupervised Facial Attribute Editing via Subspace Learning

Bo Liu; Xuan Cui; Run Zeng; Wei Duan; Chongwen Liu; Jinrui Qian; Lianggui Tang; Hongping Gan

cs.CVMar 14, 2026

U-Face: An Efficient and Generalizable Framework for Unsupervised Facial Attribute Editing via Subspace Learning

Authors: Bo Liu, Xuan Cui, Run Zeng, Wei Duan, Chongwen Liu, Jinrui Qian, Lianggui Tang, Hongping Gan

Organizations: Chongqing Key Laboratory of Intelligent Sensing and Blockchain Technology, Chongqing Technology and Business University, Chongqing, China · School of Artiﬁcial Intelligence, Chongqing Technology and Business University, Chongqing, China · School of Software, Northwestern Polytechnical University, Xi’an, China

Abstract

Latent space-based facial attribute editing methods have gained popularity in applications such as digital entertainment, virtual avatar creation, and human-computer interaction systems due to their potential for efficient and flexible attribute manipulation, particularly for continuous edits. Among these, unsupervised latent space-based methods, which discover effective semantic vectors without relying on labeled data, have attracted considerable attention in the research community. However, existing methods still encounter difficulties in disentanglement, as manipulating a specific facial attribute may unintentionally affect other attributes, complicating fine-grained controllability. To address these challenges, we propose a novel framework designed to offer an effective and adaptable solution for unsupervised facial attribute editing, called Unsupervised Facial Attribute Controllable Editing (U-Face). The proposed method frames semantic vector learning as a subspace learning problem, where latent vectors are approximated within a lower-dimensional semantic subspace spanned by a semantic vector matrix. This formulation can also be equivalently interpreted from a projection-reconstruction perspective and further generalized into an autoencoder framework, providing a foundation that can support disentangled representation learning in a flexible manner. To improve disentanglement and controllability, we impose orthogonal non-negative constraints on the semantic vectors and incorporate attribute boundary vectors to reduce entanglement in the learned directions. Although these constraints make the optimization problem challenging, we design an alternating iterative algorithm, called Alternating Iterative Disentanglement and Controllability (AIDC), with closed-form updates and provable convergence under specific conditions.

Figures & tables

Fig. 1 : Unsupervised Facial Attribute Controllable Editing (U-Face) framework.

Fig. 2 : $\mathbf{w}_{1},\mathbf{w}_{2},\ldots,\mathbf{w}_{5}$ are the five semantic vectors. (a) shows the original facial image $\mathbf{M}_{ori}=G(\mathbf{z})$ , and (b)-(f) show edited images $\mathbf{M}_{edit}^{i}=G(\mathbf{z}+\mathbf{w}_{i}),i=1,2,\ldots,5$ . Each semantic vector simultaneously manipulates multiple facial attributes: $\mathbf{w}_{1}$ adjusts both glasses and age, $\mathbf{w}_{2}$ manipulates glasses and hairstyle, and the others similarly influence various attributes.

Panel of Figure 3 — Fig. 3 : Qualitative comparison of U-Face and GAN-based baseline methods on StyleGAN and StyleGAN2 across the Age, Gender, Hairstyle, and Smile attributes.

Fig. 4 : Qualitative evaluation of U-Face and diffusion-based baseline methods across on the Age, Gender, and Smile attributes.

Method	InterFaceGAN ( Shen et al., 2020b )					MDSE ( Naveh, 2023 )					U-Face (Our)
	Age	Gender	Hairstyle	Pose	Smile	Age	Gender	Hairstyle	Pose	Smile	Age	Gender	Hairstyle	Pose	Smile
Age	–	0.565 (0.01)	0.232 (0.01)	0.090 (0.00)	0.154 (0.02)	–	0.388 (0.03)	0.265 (0.01)	0.008 (0.00)	0.120 (0.01)	–	0.311 (0.02)	0.211 (0.03)	0.037 (0.00)	0.017 (0.00)
Gender	0.565 (0.01)	–	0.268 (0.01)	0.017 (0.00)	0.198 (0.02)	0.388 (0.03)	–	0.212 (0.00)	0.029 (0.00)	0.050 (0.00)	0.311 (0.02)	–	0.191 (0.01)	0.032 (0.00)	0.015 (0.00)
Hairstyle	0.232 (0.01)	0.268 (0.01)	–	0.302 (0.02)	0.254 (0.02)	0.265 (0.01)	0.212 (0.00)	–	0.156 (0.01)	0.033 (0.00)	0.211 (0.03)	0.191 (0.01)	–	0.111 (0.01)	0.015 (0.00)
Pose	0.090 (0.00)	0.017 (0.00)	0.302 (0.02)	–	0.098 (0.01)	0.008 (0.00)	0.029 (0.00)	0.156 (0.01)	–	0.018 (0.00)	0.037 (0.00)	0.032 (0.00)	0.111 (0.01)	–	0.025 (0.00)
Smile	0.154 (0.02)	0.198 (0.02)	0.254 (0.02)	0.098 (0.01)	–	0.120 (0.01)	0.050 (0.00)	0.033 (0.00)	0.018 (0.00)	–	0.017 (0.00)	0.015 (0.00)	0.015 (0.00)	0.025 (0.00)	–

Table 1: Correlation matrix for InterFaceGAN, MDSE and U-Face across different facial attributes. Standard deviations are shown in parentheses. Avg represents the average value of each column, excluding diagonal values.

Method	GANSpace ( Härkönen et al., 2020 )					SeFa ( Shen and Zhou, 2021 )					U-Face(Our)
	Age	Gender	Hairstyle	Pose	Smile	Age	Gender	Hairstyle	Pose	Smile	Age	Gender	Hairstyle	Pose	Smile
Age	–	0.715 (0.01)	0.309 (0.01)	0.116 (0.01)	0.125 (0.01)	–	0.533 (0.01)	0.366 (0.01)	0.039 (0.01)	0.001 (0.00)	–	0.311 (0.02)	0.211 (0.03)	0.037 (0.00)	0.017 (0.00)
Gender	0.715 (0.01)	–	0.546 (0.01)	0.177 (0.01)	0.037 (0.00)	0.533 (0.01)	–	0.281 (0.01)	0.133 (0.01)	0.170 (0.01)	0.311 (0.02)	–	0.191 (0.01)	0.032 (0.00)	0.015 (0.00)
Hairstyle	0.309 (0.01)	0.546 (0.01)	–	0.234 (0.01)	0.202 (0.01)	0.366 (0.01)	0.281 (0.01)	–	0.211 (0.01)	0.180 (0.01)	0.211 (0.03)	0.191 (0.01)	–	0.111 (0.01)	0.015 (0.00)
Pose	0.116 (0.01)	0.177 (0.01)	0.234 (0.01)	–	0.153 (0.01)	0.039 (0.01)	0.133 (0.01)	0.211 (0.01)	–	0.150 (0.01)	0.037 (0.00)	0.032 (0.00)	0.111 (0.01)	–	0.025 (0.00)
Smile	0.125 (0.01)	0.037 (0.00)	0.202 (0.01)	0.153 (0.01)	–	0.001 (0.00)	0.170 (0.01)	0.180 (0.01)	0.150 (0.01)	–	0.017 (0.00)	0.015 (0.00)	0.015 (0.00)	0.025 (0.00)	–

Table 2: Correlation matrix for GANSpace, SeFa and U-Face across different facial attributes. Standard deviations are shown in parentheses. Avg represents the average value of each column, excluding diagonal values.

Fig. 5 : Qualitative evaluation of U-Face in controllable editing for Age, Smile, and Hairstyle attributes. When the editing magnitude $\beta$ is positive (e.g., $\beta=0.3$ ) or negative (e.g., $\beta=-0.3$ ), the edited images exhibit significantly changes.

Fig. 6 : Qualitative comparison of U-Face in controllable editing for the Gender attribute. On the left is the transformation from male to female and on the right is the transformation from female to male. When the editing magnitude increases, the change in the Gender attribute becomes more significant.

Model	Attribute	Supervised Method						Unsupervised Method
		InterFaceGAN ( Shen et al., 2020b )		EnjoyGAN ( Zhuang et al., 2021 )		MDSE ( Naveh, 2023 )		GANSpace ( Härkönen et al., 2020 )		SeFa ( Shen and Zhou, 2021 )		AdaTrans ( Huang et al., 2023 )		SDFlow ( Li et al., 2024 )		U-Face (Our)
		FID $\downarrow$	LPIPS $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$
StyleGAN	Age	68.93 (0.03)	0.3333 (0.02)	74.32 (0.02)	0.3856 (0.03)	87.54 (0.01)	0.4166 (0.02)	64.86 (0.02)	0.3119 (0.01)	60.91 (0.03)	0.2933 (0.02)	62.34 (0.02)	0.2644 (0.03)	59.39 (0.02)	0.2425 (0.01)	45.72 (0.01)	0.2387 (0.02)
	Gender	82.38 (0.03)	0.3814 (0.02)	76.32 (0.02)	0.3611 (0.03)	76.99 (0.01)	0.3825 (0.02)	71.99 (0.02)	0.3741 (0.01)	70.79 (0.03)	0.3428 (0.02)	78.59 (0.02)	0.3323 (0.03)	68.08 (0.02)	0.2641 (0.01)	55.74 (0.01)	0.2156 (0.02)
	Hairstyle	68.63 (0.01)	0.3154 (0.02)	75.66 (0.02)	0.3813 (0.03)	–	–	38.05 (0.02)	0.1937 (0.01)	25.13 (0.01)	0.1448 (0.02)	72.55 (0.03)	0.3537 (0.02)	66.5 (0.02)	0.2536 (0.01)	43.21 (0.02)	0.2322 (0.02)
	Pose	75.65 (0.00)	0.4112 (0.02)	–	–	73.11 (0.01)	0.3548 (0.03)	53.95 (0.02)	0.2526 (0.01)	65.63 (0.03)	0.3721 (0.02)	–	–	–	–	41.57 (0.01)	0.2213 (0.02)

Table 3: FID and LPIPS comparisons of different methods on three pre-trained GAN Models. "–" indicates that the method does not provide implementation details for editing this facial attribute. Standard deviations are shown in parentheses.

Model	Attribute	Supervised Method						Unsupervised Method
		InterFaceGAN ( Shen et al., 2020b )		EnjoyGAN ( Zhuang et al., 2021 )		MDSE ( Naveh, 2023 )		GANSpace ( Härkönen et al., 2020 )		SeFa ( Shen and Zhou, 2021 )		AdaTrans ( Huang et al., 2023 )		SDFlow ( Li et al., 2024 )		U-Face (Our)
		IS $\uparrow$	SSIM $\uparrow$	IS $\uparrow$	SSIM $\uparrow$	IS $\uparrow$	SSIM $\uparrow$	IS $\uparrow$	SSIM $\uparrow$	IS $\uparrow$	SSIM $\uparrow$	IS $\uparrow$	SSIM $\uparrow$	IS $\uparrow$	SSIM $\uparrow$	IS $\uparrow$	SSIM $\uparrow$
StyleGAN	Age	2.942 (0.01)	0.6834 (0.01)	2.975 (0.01)	0.7148 (0.02)	3.014 (0.02)	0.7212 (0.01)	3.081 (0.01)	0.7036 (0.01)	2.894 (0.02)	0.7348 (0.01)	2.421 (0.01)	0.7434 (0.01)	2.458 (0.02)	0.7221 (0.01)	3.391 (0.01)	0.7726 (0.02)
	Gender	3.114 (0.01)	0.6331 (0.01)	3.113 (0.02)	0.7240 (0.01)	3.242 (0.01)	0.7644 (0.02)	2.854 (0.01)	0.6549 (0.01)	3.275 (0.01)	0.6789 (0.02)	2.410 (0.01)	0.6824 (0.01)	2.534 (0.02)	0.6930 (0.01)	3.301 (0.01)	0.7344 (0.02)
	Hairstyle	3.062 (0.01)	0.7146 (0.01)	2.792 (0.02)	0.7123 (0.01)	–	–	3.033 (0.01)	0.8235 (0.01)	3.101 (0.01)	0.7847 (0.02)	2.553 (0.01)	0.6627 (0.01)	2.454 (0.02)	0.7069 (0.01)	2.362 (0.01)	0.8324 (0.02)
	Pose	3.145 (0.01)	0.6131 (0.01)	–	–	2.943 (0.02)	0.7879 (0.01)	3.091 (0.01)	0.7569 (0.01)	3.230 (0.01)	0.6445 (0.02)	–	–	–	–	3.263 (0.01)	0.7986 (0.01)

Table 4: IS and SSIM comparisons of different methods on three pre-trained GAN Models. "–" indicates that the method does not provide implementation details for editing this facial attribute. Standard deviations are shown in parentheses.

Attribute	Diffusion-based Method									GAN-based Method (StyleGAN Backbone)
	DDS ( Hertz et al., 2023 )			CDS ( Nam et al., 2024 )			FPE ( Liu et al., 2024a )			U-Face (Our)
	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$	FID $\downarrow$	LPIPS $\downarrow$
Age	0.7931 (0.03)	86.34 (0.05)	0.1721 (0.01)	0.7121 (0.00)	70.28 (0.01)	0.2802 (0.02)	0.6823 (0.02)	64.36 (0.00)	0.2726 (0.02)	0.7726 (0.02)	45.72 (0.01)	0.2387 (0.02)
Gender	0.4231 (0.01)	83.57 (0.03)	0.4122 (0.00)	0.6946 (0.02)	69.17 (0.03)	0.3024 (0.03)	0.3944 (0.05)	72.43 (0.00)	0.4936 (0.03)	0.7344 (0.02)	55.74 (0.01)	0.2156 (0.02)
Smile	0.5234 (0.01)	83.36 (0.00)	0.2853 (0.02)	0.7647 (0.00)	80.48 (0.01)	0.2159 (0.01)	0.5936 (0.00)	73.98 (0.02)	0.2710 (0.00)	0.8913 (0.02)	39.41 (0.01)	0.1231 (0.02)
Avg.	0.5799 (0.01)	84.42 (0.02)	0.2899 (0.01)	0.7238 (0.00)	73.31 (0.01)	0.2662 (0.02)	0.5568 (0.01)	70.26 (0.00)	0.3457 (0.01)	0.7994 (0.02)	46.96 (0.01)	0.1925 (0.02)

Table 5: A comparison between our proposed U-Face and state-of-the-art diffusion-based facial attribute editing methods in terms of SSIM, FID, LPIPS, and inference time, where U-Face is tested on the StyleGAN backbone. Standard deviations are shown in parentheses.

Variant	Avg corr $\downarrow$	FID $\downarrow$	LPIPS $\downarrow$	SSIM $\uparrow$
w/o boundary ( $\lambda{=}0$ )	0.274	52.34	0.2234	0.7063
w/o non-negativity ( $\alpha{=}0$ )	0.155	50.34	0.2466	0.7938
w/o orthogonality	0.221	51.22	0.2214	0.7541
U - Face (full)	0.096	45.13	0.2062	0.8059

Table 6: Ablation study on U - Face components (averaged over selected attributes). Lower corr/FID/LPIPS is better, higher SSIM is better.

Fig. 7 : Comparison of the sensitivity for the regularization parameter $\lambda$ in AIDC using the average values of FID, LPIPS, IS, and SSIM . For $\alpha$ values of $\{0.1,0.2,0.5,1,2,3,5\}$ , $\lambda$ is varied over the range $\{1,2,4,5\}$ .

Fig. 8 : Comparison of the sensitivity for the regularization parameter $\alpha$ in AIDC using the average values of FID, LPIPS, IS, and SSIM . For $\lambda$ values of $\{0.1,0.2,0.5,1,2,3,5\}$ , $\alpha$ is varied over the range $\{1,2,4,5\}$ .

Scenario	I/O	User controls	Practical notes
Interactive entertainment (game/NPC authoring)	Input: designer seed or style assets; Output: edited portraits/textures	Sliders for Age, Gender, Hairstyle, Smile; $\beta\in[-0.3,0.3]$	Offline authoring; disentangled edits avoid side effects; export to DCC/engine pipelines.
Digital avatars (meeting/social apps)	Input: selfie or short video; Output: avatar variants	Interactive sliders with preview; presets	Monitor identity similarity; cap edits if drift is large; latency per Table 5 .
Virtual makeup / try-on	Input: photo; Output: before/after grid or slider preview	Smile intensity, Hairstyle volume, Age softening ( $\beta$ maps to intensity)	Reduced entanglement keeps skin/texture consistent; provide reset/original toggle.

Table 7: Application summary. U-Face exposes attribute "sliders” (

\beta

) for controllable edits while monitoring identity drift.

Panel of Figure 16 — Fig. 9 : Visualization of Limitations on StyleGAN by U-Face.

Appendix figures & tables1 asset

Supplementary material from the paper’s appendix.

Appendix

Fig. 1 : Comparison of identity consistency preservation across SeFa, GANSpace, EnjoyGAN, InterFaceGAN, and U-Face using the $IDS$ metric for the Age, Gender, Smile, and Hairstyle attributes. The editing magnitude $\beta$ varies from $0.1$ to $0.8$ .

Image Editing

Image editing changes selected visual content while preserving the parts of an image that should remain unchanged.

See all 315 papers

Apr 22, 2026

MD-Face: MoE-Enhanced Label-Free Disentangled Representation for Interactive Facial Attribute Editing

GAN-based facial attribute editing is widely used in virtual avatars and social media but often suffers from attribute entanglement, where modifying one face attribute unintentionally alters others. While supervised disentangled representation learning can address this, it relies heavily on labeled data, incurring high annotation costs. To address these challenges, we propose MD-Face, a label-free disentangled representation learning framework based on Mixture of Experts (MoE). MD-Face utilizes a MoE backbone with a gating mechanism that dynamically allocates experts, enabling the model to learn semantic vectors with greater independence. To further enhance attribute entanglement, we introduce a geometry-aware loss, which aligns each semantic vector with its corresponding Semantic Boundary Vector (SBV) through a Jacobian-based pushforward method. Experiments with ProGAN and StyleGAN show that MD-Face outperforms unsupervised baselines and competes with supervised ones. Compared to diffusion-based methods, it offers better image quality and lower inference latency, making it ideal for interactive editing.

Xuan Cui, Yunfei Zhao, Bo Liu +2

Disentangled Representation Learning Unsupervised Representation Learning Image Editing Generative Adversarial Networks

Ask PDF

Apr 23, 2026

AttDiff-GAN: A Hybrid Diffusion-GAN Framework for Facial Attribute Editing

Facial attribute editing aims to modify target attributes while preserving attribute-irrelevant content and overall image fidelity. Existing GAN-based methods provide favorable controllability, but often suffer from weak alignment between style codes and attribute semantics. Diffusion-based methods can synthesize highly realistic images; however, their editing precision is limited by the entanglement of semantic directions among different attributes. In this paper, we propose AttDiff-GAN, a hybrid framework that combines GAN-based attribute manipulation with diffusion-based image generation. A key challenge in such integration lies in the inconsistency between one-step adversarial learning and multi-step diffusion denoising, which makes effective optimization difficult. To address this issue, we decouple attribute editing from image synthesis by introducing a feature-level adversarial learning scheme to learn explicit attribute manipulation, and then using the manipulated features to guide the diffusion process for image generation, while also removing the reliance on semantic direction-based editing. Moreover, we enhance style-attribute alignment by introducing PriorMapper, which incorporates facial priors into style generation, and RefineExtractor, which captures global semantic relationships through a Transformer for more precise style extraction. Experimental results on CelebA-HQ show that the proposed method achieves more accurate facial attribute editing and better preservation of non-target attributes than state-of-the-art methods in both qualitative and quantitative evaluations.

Wenmin Huang, Weiqi Luo, Xiaochun Cao +1

GuangDong Province Key Lab of Information Security Technology and School of Computer Science and Engineering, Sun Yat-sen University, Guangdong 510000, China · School of Cyber Science and Technology, Shenzhen Campus, Sun Yat-sen University, Shenzhen 518107, China · Guangdong Laboratory of Machine Perception and Intelligent Computing, Faculty of Engineering, Shenzhen MSU-BIT University, Shenzhen, 518116, China

Diffusion Model Guidance Diffusion-Based Image Editing Image Editing Diffusion Models

Ask PDF

Feb 22, 2026

Controlled Face Manipulation and Synthesis for Data Augmentation

Deep learning vision models excel with abundant supervision, but many applications face label scarcity and class imbalance. Controllable image editing can augment scarce labeled data, yet edits often introduce artifacts and entangle non-target attributes. We study this in facial expression analysis, targeting Action Unit (AU) manipulation where annotation is costly and AU co-activation drives entanglement. We present a facial manipulation method that operates in the semantic latent space of a pre-trained face generator (Diffusion Autoencoder). Using lightweight linear models, we reduce entanglement of semantic features via (i) dependency-aware conditioning that accounts for AU co-activation, and (ii) orthogonal projection that removes nuisance attribute directions (e.g., glasses), together with an expression neutralization step to enable absolute AU edit. We use these edits to balance AU occurrence by editing labeled faces and to diversify identities/demographics via controlled synthesis. Augmenting AU detector training with the generated data improves accuracy and yields more disentangled predictions with fewer co-activation shortcuts, outperforming alternative data-efficient training strategies and suggesting improvements similar to what would require substantially more labeled data in our learning-curve analysis. Compared to prior methods, our edits are stronger, produce fewer artifacts, and preserve identity better.

Joris Kirchner, Amogh Gudi, Marian Bittner +1

Vicarious Perception Technologies (VicarVision) · Delft University of Technology

Synthetic Data Augmentation Image Editing Controllable Image Generation Facial Action Unit Detection Disentangled Representation Learning

Ask PDF

Abstract

Image Editing

Explore similar work

MD-Face: MoE-Enhanced Label-Free Disentangled Representation for Interactive Facial Attribute Editing

AttDiff-GAN: A Hybrid Diffusion-GAN Framework for Facial Attribute Editing

Controlled Face Manipulation and Synthesis for Data Augmentation