Knowledge Priors for Identity-Disentangled Open-Set Privacy-Preserving Video FER

Feng Xu; Xun Li; Lars Petersson; Yulei Sui; David Ahmedt-Aristizabal; Dadong Wang

cs.CVMar 22, 2026

Knowledge Priors for Identity-Disentangled Open-Set Privacy-Preserving Video FER

Authors: Feng Xu, Xun Li, Lars Petersson, Yulei Sui, David Ahmedt-Aristizabal, Dadong Wang

Organizations: School of CSE, UNSW Sydney, Australia · CSIRO’s Data61, Australia

Abstract

Facial expression recognition relies on facial data that inherently expose identity and thus raise significant privacy concerns. Current privacy-preserving methods typically fail in realistic open-set video settings where identities are unknown, and identity labels are unavailable. We propose a two-stage framework for video-based privacy-preserving FER in challenging open-set settings that requires no identity labels at any stage. To decouple privacy and utility, we first train an identity-suppression network using intra- and inter-video knowledge priors derived from real-world videos without identity labels. This network anonymizes identity while preserving expressive cues. A subsequent denoising module restores expression-related information and helps recover FER performance. Furthermore, we introduce a falsification-based validation method that uses recognition priors to rigorously evaluate privacy robustness without requiring annotated identity labels. Experiments on three video datasets demonstrate that our method effectively protects privacy while maintaining FER accuracy comparable to identity-supervised baselines.

Figures & tables

Fig. 1 : Our framework relies solely intra- and inter-video knowledge priors extracted from videos based on video characteristics to learn privacy preservation transformations without requiring identity labels. It integrates a denoising step to recover FER performance after anonymization and introduces a falsification-based validation scheme for privacy assessment.

Privacy Preservation	Identity Label	Denoise	Utility Task	Privacy Validation	Validation Require Label	Scenario
[ 16 ]	Yes	No	Act. Recog.	Yes	Yes	Closed
[ 4 ] [ 7 ]	No	No	Act. Recog.	Yes	Yes	Open
Blurring and [ 18 ]	No	No	N/A	No	N/A	Open
[ 17 ]	Yes	Yes	FER	Yes	Yes	Closed
Ours	No	Yes	FER	Yes	No	Open

TABLE I : Comparison of our approach with existing privacy-preserving approaches with their associated utility tasks.

Fig. 2 : Overview of our privacy-preserving video-based FER framework. (1) Face detection and tracking extract cropped face sequences and assign pseudo-identity labels. (2) These labeled faces train the privacy-preserving reconstruction network (

f_{pp}

) using triplet and binary cross-entropy (BCE) losses, guided by the frozen identity extractor (

f_{e}

). (3)

f_{pp}

generates anonymized faces, which are enhanced by a denoising network

f_{denoise}

to support FER training via (

f_{fer}

). (4) Privacy robustness is evaluated using a falsification-based approach without identity labels, comparing original, anonymized, denoised, and reconstructed outputs via a recovery model (

f_{reco}

), with a privacy preservation ratio

P_{pre}

computed for quantitative comparison.

Fig. 3 : Illustration of inter- and intra-video knowledge priors for training $f_{pp}$ . Each video undergoes face detection, alignment, and tracking to obtain consistent tracking IDs and pseudo-labels. Expression labels (e.g., “Angry”) group semantically similar samples across videos. Training batches contain triplets (anchor, positive, negative) plus an extra face sharing the anchor’s ID to enforce identity disentanglement. A binary label (0 or 1) signals identity consistency, enabling identity-irrelevant learning without ground-truth ID labels.

GT	Rules
0	1 . Two different images from the same original video with different unique ID; 2 . Two images from two different original videos ;
1	3 . Two images from the same original video with the same unique ID;
0	4 . One image from original video and the other from the same video but privacy-preserved , with the same unique ID; 5 . One image from original video and the other from the same video but denoised privacy-preserved , with the same unique ID; 6 . One image from original video and the other from the same video but recovered privacy-preserved , with the same unique ID; 7 . One image from original video and the other from the same video but recovered denoised privacy-preserved , with the same unique ID

TABLE II : Test case generation rules. GT stands for ground truth. 0 means that

f_{match}

needs to output the pair have different identities, otherwise is 1.

Dataset	Model	Hap	Sad	Neu	Ang	Sup	Dis	Fea	Cal
DFEW	R(2+1)D	79.96	40.11	56.93	50.11	49.32	3.45	22.10	N/A
	I3D	77.91	46.17	55.99	46.21	46.26	3.45	20.99	N/A
	R(2+1)D	73.21	35.36	47.57	39.77	38.78	3.45	17.68	N/A
	I3D	68.71	39.31	48.50	40.23	39.80	3.45	16.57	N/A
CREMA-D	R(2+1)D	98.34	80.77	84.89	91.69	N/A	98.89	80.30	N/A
	I3D	99.17	82.21	87.31	90.03	N/A	93.35	77.50	N/A

TABLE III : R(2+1)D and I3D FER accuracy (%) on the testing parts of datasets without privacy preservation in regular font and post privacy preservation and denoising in bold and italics font.

Fig. 4 : Privacy preservation ratio,

P_{pre}

and FER accuracy tradeoff with 95% confidence interval.

Approach	Acc. of R(2+1)D $\uparrow$			Acc. of I3D $\uparrow$
Approach	DFEW	CREMA-D	RAVDESS	DFEW	CREMA-D	RAVDESS
N-PP	43.14	89.15	86.44	42.43	88.26	87.73
GB	35.89 (0.4)	78.54 (0.5)	77.54(0.6)	36.32 (0.4)	77.43 (0.5)	78.21(0.6)
Adver. [ 4 ]	32.44	74.59	75.08	33.32	73.42	77.31
Face S. [ 18 ]	29.31	67.56	68.39	28.92	68.36	69.85
Contr-HL [ 17 ]	N/A	79.19	79.93	N/A	78.42	77.59

TABLE IV : Privacy-preserved and Denoised FER accuracy (%). ‘N-PP’ stands for no privacy preservation , and it means standard FER task. In the GB row, the numbers in parentheses, (), represent the

\sigma

hyperparameter values in Gaussian blurring. The ‘N/A’ stands for Not Applicable.

Fig. 5 : An example video frame from the DFEW dataset.

Approach	DFEW (82,746)	CREMA-D (2466)	RAVDESS (1084)
GB	0.4100	0.2206	0.2039
Adver. [ 4 ]	0.6843	0.6363	0.7399
Face S. [ 18 ]	0.7358	0.5856	0.6780
Contr-HL [ 17 ]	N/A	0.9408	0.9419
Ours	0.9620	0.9071	0.9363

TABLE V : Privacy preservation ratio,

p_{pre}

. (): number of cases

Approach	SSIM $\downarrow$			PSNR $\downarrow$
Approach	DFEW	CREMA-D	RAVDESS	DFEW	CREMA-D	RAVDESS
GB	0.9122	0.9443	0.9534	29.43	32.85	33.91
Adver. [ 4 ]	0.5120	0.5292	0.5145	21.05	22.07	21.09
Face S. [ 18 ]	0.4372	0.4934	0.5002	15.76	16.31	15.32
Contr-HL [ 17 ]	N/A	0.4243	0.4398	N/A	11.49	12.09
Ours	0.4076	0.4521	0.4432	11.47	12.01	13.98

TABLE VI : SSIM and PSNR results.

Tracking Approach:			MTCNN+ArcFace	SAMURAI	Ours
DFEW	Num of Identity		20347	26032	25969
	Acc. (%)	R(2+1)D	36.42	36.71	36.54
	Acc. (%)	I3D	36.60	36.27	36.65
	$P_{pre}$ (%)		96.02	96.31	96.20
CREMA-D	Acc. (%)	R(2+1)D	80.85	81.21	80.90
	Acc. (%)	I3D	81.97	82.49	82.12

TABLE VII : The impact of various tracking approaches on the framework.

TABLE VIII : (a) Ablation ignoring expression labels during dataset generation; only FER accuracy and

P_{pre}

are evaluated. (b) FER accuracy and

P_{pre}

on DFEW for varying

\alpha

	DFEW	CREMA-D	RAVDESS
R(2+1)D Acc.	27.55 ( 33.98 )	71.49 (79.02)	73.54 (77.64)
I3D Acc.	28.31 ( 34.65 )	72.43 (79.21)	73.11 (79.38)
$P_{pre}$	95.01 (95.98)	87.59 (91.29)	90.13 (93.98)

TABLE IX : FER accuracy (%) and privacy preservation ratio (%),

P_{pre}

on video clips without

f_{denoise}

. Numbers in parentheses show results with

f_{denoise}

for comparison.

$k$	$m$	$n$	Batch Size
1	2	1	4
2	4	5	16
3	6	21	64
4	8	85	256
5	10	341	1024

TABLE X : Available options of batch size and number of triplets (n).

Fig. 6 : t-SNE of the original, privacy-preserved and denoised privacy-preserved identity embedding from DFEW. It indicates that the model systematically shifts identity features away from their original identities.

Fig. 7 : Distribution of IED Across Stages

Dataset		Rule 4	Rule 5	Rule 6	Rule 7	PP Ratio
DFEW	# Cases	25969	25969	25969	25969
	GB	15842	16049	5291	5411	0.4100
	Adver.	21437	21625	13975	14046	0.6843
	Face S	22837	22981	14675	14846	0.7253
	Ours	24897	25297	24681	25052	0.9620
CREMA-D	# Cases	4464	4464	4464	4464

TABLE XI : Statistics of Rule 4–7 cases, including correct validations and privacy-preservation (PP) ratios.

Fig. 8 : FFE task on DFEW dataset using R(2+1)D model without privacy preservation

Fig. 9 : FFE task on DFEW dataset using I3D model without privacy preservation

Fig. 10 : FFE task on CREMA-D dataset using R(2+1)D model without privacy preservation

Fig. 11 : FFE task on CREMA-D dataset using I3D model without privacy preservation

Fig. 12 : FFE task on RAVDESS dataset using R(2+1)D model without privacy preservation

Fig. 13 : FFE task on RAVDESS dataset using I3D model without privacy preservation

Fig. 14 : FFE task on privacy-preserved and denoised DFEW dataset using R(2+1)D model

Fig. 15 : FFE task on privacy-preserved and denoised DFEW dataset using I3D model

Fig. 16 : FFE task on privacy-preserved and denoised CREMA-D dataset using R(2+1)D model

Fig. 17 : FFE task on privacy-preserved and denoised CREMA-D dataset using I3D model

Fig. 18 : FFE task on privacy-preserved and denoised RAVDESS dataset using R(2+1)D model

Fig. 19 : FFE task on privacy-preserved and denoised RAVDESS dataset using I3D model

Facial Expression Recognition

Facial Expression Recognition studies how computers infer emotion and facial actions from images and video.

See all 79 papers

Oct 8, 2026

Expression-Diverse References for Identity-Preserving Video Generation

Identity-preserving video generation aims to maintain a subject's identity while synthesizing realistic videos. Yet a single reference portrait captures the subject's appearance under only one facial configuration. As expressions change, facial appearance can vary in highly identity-specific ways, leaving the subject's appearance under unseen expressions underdetermined by the reference alone. This expression-dependent variation also complicates evaluation: similarity to a neutral reference may decrease under strong expressions even for real images of the same person. We investigate this limitation from both generation and evaluation perspectives. First, we quantify how face-recognition similarity varies with expression intensity using controlled photographs and MEAD videos. We then construct a compact yet expressive reference gallery that captures diverse expression-dependent facial configurations. Matching against this gallery provides a more robust measure of identity similarity under expressive motion. To further expose performance degradation with expression intensity, we report identity similarity separately for mild, intense, and extreme expressions. For generation, we extend Stand-In to condition on our expression-diverse reference sets and develop a data-curation pipeline that extracts consistent yet diverse face crops from training videos. In practical settings where only a single portrait is available, we construct the reference set by synthesizing additional expressions with a pretrained facial reenactment model. On our controlled benchmark, both real and synthesized reference sets outperform the evaluated baselines in identity similarity across all three expression-intensity regimes, with the largest improvements for extreme expressions.

Tianwen Fu, Wenbin Teng, Gonglin Chen +3

Institute for Creative Technologies, University of Southern California

Identity-Preserving Video Generation Video Generation Evaluation

Ask PDF

Jul 30, 2026

Private Face Recognition Training Dataset Publication via Identity-Decoupled and Geometry-Preserving Face Distillation

Publishing private face recognition~(FR) training datasets is privacy-sensitive because faces expose identity information. Private FR training dataset publication mitigates this risk by releasing protected proxies as substitutes for private training faces. However, training FR models with such data introduces an identity paradox: the identity cues that make released faces useful for recognition supervision are also the cues that make them linkable to real individuals. A protected face should be decoupled from the original identity, yet still behave as a reliable identity sample for training. Removing these cues too aggressively may destroy the class structure needed for recognition learning, whereas preserving them too faithfully may increase source-identity linkability. We argue that this paradox stems from conflating source-aligned identity semantics with recognition-useful proxy identity geometry. The former should be suppressed to reduce linkage to private individuals, while the latter should be preserved for FR learning. Based on this insight, we propose Private Face Distillation, an identity-decoupling and geometry-preserving framework. It uses Orthogonal Geometry Preservation to construct decoupled proxy identities from private identity representations while maintaining hyperspherical geometry, and Relational Topology Alignment to preserve identity relations for recognition learning. Experiments across multiple domain-shifted FR scenarios show that Private Face Distillation achieves stronger utility than the evaluated publication baselines. On IJB-C surveillance, it improves

\mathrm{TAR}@\mathrm{FAR}{=}1\text{e-}{3}

by 3.94% over the baseline while reducing source-identity linkability. These results suggest that private FR training dataset publication should decouple source-identity correspondence while preserving proxy identity geometry.

Shuhuan Chen, Xiangyu Zhu, Weisong Zhao +6

1IIE, CAS · 2CBSR&MAIS, CASIA · 4SAI, UCAS +3

Face Recognition Knowledge Distillation

Ask PDF

Mar 18, 2026

Face anonymization preserving facial expressions and photometric realism

The widespread sharing of face images on social media platforms and in large-scale datasets raises pressing privacy concerns, as biometric identifiers can be exploited without consent. Face anonymization seeks to generate realistic facial images that irreversibly conceal the subject's identity while preserving their usefulness for downstream tasks. However, most existing generative approaches focus on identity removal and image realism, often neglecting facial expressions as well as photometric consistency -- specifically attributes such as illumination and skin tone -- that are critical for applications like relighting, color constancy, and medical or affective analysis. In this work, we propose a feature-preserving anonymization framework that extends DeepPrivacy by incorporating dense facial landmarks to better retain expressions, and by introducing lightweight post-processing modules that ensure consistency in lighting direction and skin color. We further establish evaluation metrics specifically designed to quantify expression fidelity, lighting consistency, and color preservation, complementing standard measures of image realism, pose accuracy, and re-identification resistance. Experiments on the CelebA-HQ dataset demonstrate that our method produces anonymized faces with improved realism and significantly higher fidelity in expression, illumination, and skin tone compared to state-of-the-art baselines. These results underscore the importance of feature-aware anonymization as a step toward more useful, fair, and trustworthy privacy-preserving facial data.

Luigi Celona, Simone Bianco, Raimondo Schettini

Department of Informatics, Systems and Communication, University of Milano-Bicocca, 20126 Milan, Italy

Privacy-Preserving ML

Ask PDF

Abstract

Facial Expression Recognition

Explore similar work

Expression-Diverse References for Identity-Preserving Video Generation

Private Face Recognition Training Dataset Publication via Identity-Decoupled and Geometry-Preserving Face Distillation

Face anonymization preserving facial expressions and photometric realism