A2BFR: Attribute-Aware Blind Face Restoration

Chenxin Zhu; Yushun Fang; Lu Liu; Shibo Yin; Xiaohong Liu; Xiaoyun Zhang; Qiang Hu; Guangtao Zhai

cs.CVMar 31, 2026

A2BFR: Attribute-Aware Blind Face Restoration

Authors: Chenxin Zhu, Yushun Fang, Lu Liu, Shibo Yin, Xiaohong Liu, Xiaoyun Zhang, Qiang Hu, Guangtao Zhai

Organizations: Shanghai Jiao Tong University, Shanghai, China · Xiaohongshu Inc., Shanghai, China

Abstract

Blind face restoration (BFR) aims to recover high-quality facial images from degraded inputs, yet its inherently ill-posed nature leads to ambiguous and uncontrollable solutions. Recent diffusion-based BFR methods improve perceptual quality but remain uncontrollable, whereas text-guided face editing enables attribute manipulation without reliable restoration. To address these issues, we propose A $^2$ BFR, an attribute-aware blind face restoration framework that unifies high-fidelity reconstruction with prompt-controllable generation. Built upon a Diffusion Transformer backbone with unified image-text cross-modal attention, A $^2$ BFR jointly conditions the denoising trajectory on both degraded inputs and textual prompts. To inject semantic priors, we introduce attribute-aware learning, which supervises denoising latents using facial attribute embeddings extracted by an attribute-aware encoder. To further enhance prompt controllability, we introduce semantic dual-training, which leverages the pairwise attribute variations in our newly curated AttrFace-90K dataset to enforce attribute discrimination while preserving fidelity. Extensive experiments demonstrate that A $^2$ BFR achieves state-of-the-art performance in both restoration fidelity and instruction adherence, outperforming diffusion-based BFR baselines by -0.0467 LPIPS and +52.58% attribute accuracy, while enabling fine-grained, prompt-controllable restoration even under severe degradations.

Figures & tables

Figure 1: Left: Prompt-guided specific restoration avoids uncontrolled solutions in ill-posed BFR. Middle: Compared with restore-only and restore-then-edit pipelines, A ${}^{\text{2}}$ BFR achieves faithful yet attribute-aligned restoration. Right: Quantitative results show that A ${}^{\text{2}}$ BFR simultaneously achieves superior fidelity and attribute accuracy, outperforming all compared restoration and editing methods.

Datasets	Size	Attributes	Fine-grained Captions	Construction Pipeline
SeqDeepFake [ 37 ]	49,920	5	✗	Talk-to-Edit
SEED [ 66 ]	91,526	6	✗	UltraEdit, SDXL…
AttrFace-90K	180,424	12	✓	FlowEdit + SD3.5

Table 1: Comparison of facial editing datasets. AttrFace-90K provides the largest scale, the most diverse attributes, and the most powerful construction pipeline among existing datasets.

Figure 2: AttrFace-90K overview. (a) Data construction pipeline: starting from HQ faces (FFHQ/ReFace-HQ/CelebA-HQ), we extract features, classify attributes, build prompts with Qwen3, and perform attribute editing with FlowEdit+SD3.5, followed by post-checks. (b) Attribute distribution: percentage of each controllable attribute across 90K pairs. (c) Image pair examples: source–target pairs for typical attributes.

Figure 3: Overview of our $\text{A}^{\text{2}}$ BFR. Left: The semantic dual-training (SDT) strategy pairs a single LQ input with two GT images driven by different attribute prompts, and enforces maximal separation between their latent representations in the semantic space. This encourages the model to learn attribute-discriminative, prompt-aligned behaviors. Right: At inference time, $\text{A}^{\text{2}}$ BFR takes an LQ image and a user-specified prompt, and generates an attribute-aligned, high-quality restoration through denoising process.

AttrFace-90K-Test
Metrics/Methods	GFPGAN	CodeFormer	VQFR	DR2	DifFace	DAEFR	VSPBFR	StableSR	DiT4SR	$\text{A}^{\text{2}}$ BFR
$\spadesuit$ IDS [ 6 ] $\uparrow$	0.3242	0.3337	0.3030	0.2782	0.2852	0.3012	0.2995	0.3289	0.3048	0.3495
$\spadesuit$ FID [ 14 ] $\downarrow$	20.62	20.31	18.46	43.92	27.22	23.61	26.82	25.91	18.10	18.69
$\spadesuit$ LPIPS [ 60 ] $\downarrow$	0.3070	0.3154	0.3123	0.3676	0.3398	0.2934	0.3585	0.3396	0.3442	0.2679
$\heartsuit$ HyperIQA [ 40 ] $\uparrow$	0.7380	0.7160	0.7156	0.6419	0.6127	0.7365	0.6815	0.6953	0.7609	0.8084
$\heartsuit$ TopIQ [ 3 ] $\uparrow$	0.7956	0.7740	0.7482	0.5929	0.5938	0.7760	0.7061	0.7393	0.8499	0.8748

Table 2: Quantitative results of restoration quality comparison on AttrFace-90K-Test, CelebRef-HQ-Test [ 25 ] , LFW-Test [ 15 ] , Wider-Test [ 53 ] , and WebPhoto-Test [ 49 ] . Best and second best performance are highlighted.

\uparrow

means higher is better,

\downarrow

means lower is better.

\spadesuit

denotes reference-based fidelity metrics, while

\heartsuit

represents no-reference quality metrics.

Figure 4: Qualitative comparison of restoration quality on CelebRef-HQ-Test . A ${}^{\text{2}}$ BFR restores sharper textures and more natural facial details while preserving better fidelity, surpassing diffusion- and diffusion-based baselines.

	Attribute		Fidelity				Quality			Human
Methods/Metrics	SC $\uparrow$	AA $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	IDS $\uparrow$	CP-IDS $\uparrow$	HyperIQA $\uparrow$	MANIQA $\uparrow$	MUSIQ $\uparrow$	Pref. (%)
StableSR [ 47 ]	92.49	0.3278	0.3330	21.64	0.2830	0.7619	0.6948	0.5178	71.35	2.4
+ Step1X-Edit [ 29 ]	93.73	0.7757	0.3889	23.93	0.2052	0.5331	0.7491	0.6030	74.71	2.4
+ Flux.1-Kontext [ 22 ]	94.74	0.8395	0.4726	36.36	0.1698	0.3546	0.6152	0.4648	72.81	4.3
+ Qwen-Image-Edit [ 51 ]	94.64	0.7490	0.4573	35.16	0.1577	0.3059	0.5718	0.4196	73.47	10.2
DiT4SR [ 9 ]	93.97	0.5714	0.3649	24.69	0.2916	0.5630	0.7595	0.6525	76.33	7.3

Table 3: Quantitative results of attribute alignment comparison on AttrFace-90K-Test . Best and second best performance are highlighted.

Figure 5: Qualitative comparison of attribute alignment on AttrFace-90K-Test . The top row shows one-stage restoration results, while the second and third rows display the results of the two-stage restore-then-edit pipeline. A ${}^{\text{2}}$ BFR (bottom) achieves better visual quality with accurate attribute control under the provided prompts.

Figure 6: Customizable restoration outputs of A ${}^{\text{2}}$ BFR. Each column showcases diverse results generated from the same input image by applying different facial attributes, demonstrating the model’s ability to perform user-directed BFR. The “Template Prompt” presents “A photo of a human face”, while * denotes out-of-domain attributes.

Components			Attribute		Fidelity		Quality
AAL	SDT w/o $\mathcal{L}_{\text{dual}}$	SDT w $\mathcal{L}_{\text{dual}}$	SC $\uparrow$	AA $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	HyperIQA $\uparrow$	MANIQA $\uparrow$	MUSIQ $\uparrow$
✗	✗	✗	94.59	0.4078	0.2948	25.21	0.7600	0.5892	74.43
✓	✗	✗	94.80	0.6214	0.2954	22.17	0.7561	0.6439	75.64
✗	✓	✗	94.63	0.6417	0.2889	23.76	0.7829	0.6572	75.91
✓	✓	✗	94.77	0.7757	0.2926	22.84	0.7842	0.6505	75.94
✓	✗	✓	94.70	0.8536	0.2863	21.52	0.7908	0.6628	76.22

Table 4: Ablation study of AAL and SDT on AttrFace-90K-Test . Bold indicates the best performance and underline indicates the second best.

Figure 7: Qualitative results of ablation study. It can be observed that the responsiveness to attribute texts is progressively enhanced as more components are activated.

	Attribute		Fidelity		Quality
Methods/Metrics	SC $\uparrow$	AA $\uparrow$	LPIPS $\downarrow$	FID $\downarrow$	HyperIQA $\uparrow$	MANIQA $\uparrow$	MUSIQ $\uparrow$
DiT4SR† (FFHQ)	94.23	0.4730	0.3570	25.79	0.6826	0.5283	71.33
DiT4SR† (AttrFace)	94.36	0.6308	0.3189	21.30	0.7069	0.5867	71.53
$\text{A}^{\text{2}}$ BFR (Ours)	94.70	0.8536	0.2863	21.52	0.7908	0.6628	76.22

Table 5: Quantitative comparison of face restoration models fine-tuned on FFHQ and AttrFace-90K datasets.

Geometry-Guided Image Restoration

See all 16 papers

Bridging Information Asymmetry: A Hierarchical Framework for Deterministic Blind Face Restoration
Jan 27, 2026Zhengjian Yao, Jiakui Hu, Kaiwen Li +5Diffusion Model Alignment Diffusion Models for Image Restoration
Biomedical Engineering Department, College of Future Technology, Peking University, Beijing, China
GeoMAR: Unleashing Geometrically Aligned Features for Masked Autoregressive Blind Face Restoration
Aug 4, 2026Lu Gan, Hanyu Yan, Chaofeng Chen +2Image Restoration Autoregressive Modeling
School of Artificial Intelligence, Sun Yat-sen University, Zhuhai, China
HDRFace: Rethinking Face Restoration with High-Dimensional Representation
May 14, 2026Zirui Wang, Xianhui Lin, Yi Dong +7Diffusion Models for Image Restoration Diffusion Model Guidance
City University of Hong Kong · vivo BlueImage Lab, vivo Mobile Communication Co., Ltd

Jan 27, 2026J/K move · Enter open · S save

Bridging Information Asymmetry: A Hierarchical Framework for Deterministic Blind Face Restoration

Zhengjian Yao, Jiakui Hu, Kaiwen Li, Hangzhou He+4

Biomedical Engineering Department, College of Future Technology, Peking University, Beijing, China

Diffusion Model Alignment Diffusion Models for Image Restoration Image Restoration

Open paper AskPDF arXiv

Blind face restoration remains a persistent challenge due to the inherent ill-posedness of reconstructing holistic structures from severely constrained observations. Current generative paradigms, while capable of synthesizing realistic facial details, remain limited by the under-constrained nature of blind restoration, where severely degraded inputs can be mapped to plausible yet identity-inconsistent outputs. To address this issue, we present Pref-Restore, a hierarchical framework for deterministic BFR. Our design is organized around three complementary principles: (1) Semantic Information Augmentation, where an auto-regressive semantic branch converts image and text cues into structured tokens that provide a stable high-level anchor; (2) Texture-level Fidelity Alignment, where the diffusion generator is trained under this anchor to recover identity-relevant details; and (3) Fidelity-constrained Preference Optimization, where a face-aware reward refines the diffusion trajectory while controlling the quality-fidelity trade-off. Extensive experiments on synthetic and real-world benchmarks show that Pref-Restore achieves state-of-the-art performance, with stronger identity-sensitive fidelity and lower restoration uncertainty across repeated sampling. Systematic ablations further attribute these gains to the proposed hierarchical design, showing the necessity of staged training, the robustness of the text pathway under deployment-faithful conditions, and the benefit of fidelity-constrained preference optimization.

Abstract

Geometry-Guided Image Restoration

Explore similar work

Bridging Information Asymmetry: A Hierarchical Framework for Deterministic Blind Face Restoration

GeoMAR: Unleashing Geometrically Aligned Features for Masked Autoregressive Blind Face Restoration

HDRFace: Rethinking Face Restoration with High-Dimensional Representation