Knowledge Priors for Identity-Disentangled Open-Set Privacy-Preserving Video FER

Feng Xu; Xun Li; Lars Petersson; Yulei Sui; David Ahmedt-Aristizabal; Dadong Wang

cs.CVMar 22, 2026

Knowledge Priors for Identity-Disentangled Open-Set Privacy-Preserving Video FER

Authors: Feng Xu, Xun Li, Lars Petersson, Yulei Sui, David Ahmedt-Aristizabal, Dadong Wang

Organizations: School of CSE, UNSW Sydney, Australia · CSIRO’s Data61, Australia

Abstract

Facial expression recognition relies on facial data that inherently expose identity and thus raise significant privacy concerns. Current privacy-preserving methods typically fail in realistic open-set video settings where identities are unknown, and identity labels are unavailable. We propose a two-stage framework for video-based privacy-preserving FER in challenging open-set settings that requires no identity labels at any stage. To decouple privacy and utility, we first train an identity-suppression network using intra- and inter-video knowledge priors derived from real-world videos without identity labels. This network anonymizes identity while preserving expressive cues. A subsequent denoising module restores expression-related information and helps recover FER performance. Furthermore, we introduce a falsification-based validation method that uses recognition priors to rigorously evaluate privacy robustness without requiring annotated identity labels. Experiments on three video datasets demonstrate that our method effectively protects privacy while maintaining FER accuracy comparable to identity-supervised baselines.

Figures & tables

Fig. 1 : Our framework relies solely intra- and inter-video knowledge priors extracted from videos based on video characteristics to learn privacy preservation transformations without requiring identity labels. It integrates a denoising step to recover FER performance after anonymization and introduces a falsification-based validation scheme for privacy assessment.

Privacy Preservation	Identity Label	Denoise	Utility Task	Privacy Validation	Validation Require Label	Scenario
[ 16 ]	Yes	No	Act. Recog.	Yes	Yes	Closed
[ 4 ] [ 7 ]	No	No	Act. Recog.	Yes	Yes	Open
Blurring and [ 18 ]	No	No	N/A	No	N/A	Open
[ 17 ]	Yes	Yes	FER	Yes	Yes	Closed
Ours	No	Yes	FER	Yes	No	Open

TABLE I : Comparison of our approach with existing privacy-preserving approaches with their associated utility tasks.

Fig. 2 : Overview of our privacy-preserving video-based FER framework. (1) Face detection and tracking extract cropped face sequences and assign pseudo-identity labels. (2) These labeled faces train the privacy-preserving reconstruction network (

f_{pp}

) using triplet and binary cross-entropy (BCE) losses, guided by the frozen identity extractor (

f_{e}

). (3)

f_{pp}

generates anonymized faces, which are enhanced by a denoising network

f_{denoise}

to support FER training via (

f_{fer}

). (4) Privacy robustness is evaluated using a falsification-based approach without identity labels, comparing original, anonymized, denoised, and reconstructed outputs via a recovery model (

f_{reco}

), with a privacy preservation ratio

P_{pre}

computed for quantitative comparison.

Fig. 3 : Illustration of inter- and intra-video knowledge priors for training $f_{pp}$ . Each video undergoes face detection, alignment, and tracking to obtain consistent tracking IDs and pseudo-labels. Expression labels (e.g., “Angry”) group semantically similar samples across videos. Training batches contain triplets (anchor, positive, negative) plus an extra face sharing the anchor’s ID to enforce identity disentanglement. A binary label (0 or 1) signals identity consistency, enabling identity-irrelevant learning without ground-truth ID labels.

GT	Rules
0	1 . Two different images from the same original video with different unique ID; 2 . Two images from two different original videos ;
1	3 . Two images from the same original video with the same unique ID;
0	4 . One image from original video and the other from the same video but privacy-preserved , with the same unique ID; 5 . One image from original video and the other from the same video but denoised privacy-preserved , with the same unique ID; 6 . One image from original video and the other from the same video but recovered privacy-preserved , with the same unique ID; 7 . One image from original video and the other from the same video but recovered denoised privacy-preserved , with the same unique ID

TABLE II : Test case generation rules. GT stands for ground truth. 0 means that

f_{match}

needs to output the pair have different identities, otherwise is 1.

Dataset	Model	Hap	Sad	Neu	Ang	Sup	Dis	Fea	Cal
DFEW	R(2+1)D	79.96	40.11	56.93	50.11	49.32	3.45	22.10	N/A
	I3D	77.91	46.17	55.99	46.21	46.26	3.45	20.99	N/A
	R(2+1)D	73.21	35.36	47.57	39.77	38.78	3.45	17.68	N/A
	I3D	68.71	39.31	48.50	40.23	39.80	3.45	16.57	N/A
CREMA-D	R(2+1)D	98.34	80.77	84.89	91.69	N/A	98.89	80.30	N/A
	I3D	99.17	82.21	87.31	90.03	N/A	93.35	77.50	N/A

TABLE III : R(2+1)D and I3D FER accuracy (%) on the testing parts of datasets without privacy preservation in regular font and post privacy preservation and denoising in bold and italics font.

Fig. 4 : Privacy preservation ratio,

P_{pre}

and FER accuracy tradeoff with 95% confidence interval.

Approach	Acc. of R(2+1)D $\uparrow$			Acc. of I3D $\uparrow$
Approach	DFEW	CREMA-D	RAVDESS	DFEW	CREMA-D	RAVDESS
N-PP	43.14	89.15	86.44	42.43	88.26	87.73
GB	35.89 (0.4)	78.54 (0.5)	77.54(0.6)	36.32 (0.4)	77.43 (0.5)	78.21(0.6)
Adver. [ 4 ]	32.44	74.59	75.08	33.32	73.42	77.31
Face S. [ 18 ]	29.31	67.56	68.39	28.92	68.36	69.85
Contr-HL [ 17 ]	N/A	79.19	79.93	N/A	78.42	77.59

TABLE IV : Privacy-preserved and Denoised FER accuracy (%). ‘N-PP’ stands for no privacy preservation , and it means standard FER task. In the GB row, the numbers in parentheses, (), represent the

\sigma

hyperparameter values in Gaussian blurring. The ‘N/A’ stands for Not Applicable.

Fig. 5 : An example video frame from the DFEW dataset.

Approach	DFEW (82,746)	CREMA-D (2466)	RAVDESS (1084)
GB	0.4100	0.2206	0.2039
Adver. [ 4 ]	0.6843	0.6363	0.7399
Face S. [ 18 ]	0.7358	0.5856	0.6780
Contr-HL [ 17 ]	N/A	0.9408	0.9419
Ours	0.9620	0.9071	0.9363

TABLE V : Privacy preservation ratio,

p_{pre}

. (): number of cases

Approach	SSIM $\downarrow$			PSNR $\downarrow$
Approach	DFEW	CREMA-D	RAVDESS	DFEW	CREMA-D	RAVDESS
GB	0.9122	0.9443	0.9534	29.43	32.85	33.91
Adver. [ 4 ]	0.5120	0.5292	0.5145	21.05	22.07	21.09
Face S. [ 18 ]	0.4372	0.4934	0.5002	15.76	16.31	15.32
Contr-HL [ 17 ]	N/A	0.4243	0.4398	N/A	11.49	12.09
Ours	0.4076	0.4521	0.4432	11.47	12.01	13.98

TABLE VI : SSIM and PSNR results.

Tracking Approach:			MTCNN+ArcFace	SAMURAI	Ours
DFEW	Num of Identity		20347	26032	25969
	Acc. (%)	R(2+1)D	36.42	36.71	36.54
	Acc. (%)	I3D	36.60	36.27	36.65
	$P_{pre}$ (%)		96.02	96.31	96.20
CREMA-D	Acc. (%)	R(2+1)D	80.85	81.21	80.90
	Acc. (%)	I3D	81.97	82.49	82.12

TABLE VII : The impact of various tracking approaches on the framework.

TABLE VIII : (a) Ablation ignoring expression labels during dataset generation; only FER accuracy and

P_{pre}

are evaluated. (b) FER accuracy and

P_{pre}

on DFEW for varying

\alpha

	DFEW	CREMA-D	RAVDESS
R(2+1)D Acc.	27.55 ( 33.98 )	71.49 (79.02)	73.54 (77.64)
I3D Acc.	28.31 ( 34.65 )	72.43 (79.21)	73.11 (79.38)
$P_{pre}$	95.01 (95.98)	87.59 (91.29)	90.13 (93.98)

TABLE IX : FER accuracy (%) and privacy preservation ratio (%),

P_{pre}

on video clips without

f_{denoise}

. Numbers in parentheses show results with

f_{denoise}

for comparison.

$k$	$m$	$n$	Batch Size
1	2	1	4
2	4	5	16
3	6	21	64
4	8	85	256
5	10	341	1024

TABLE X : Available options of batch size and number of triplets (n).

Fig. 6 : t-SNE of the original, privacy-preserved and denoised privacy-preserved identity embedding from DFEW. It indicates that the model systematically shifts identity features away from their original identities.

Fig. 7 : Distribution of IED Across Stages

Dataset		Rule 4	Rule 5	Rule 6	Rule 7	PP Ratio
DFEW	# Cases	25969	25969	25969	25969
	GB	15842	16049	5291	5411	0.4100
	Adver.	21437	21625	13975	14046	0.6843
	Face S	22837	22981	14675	14846	0.7253
	Ours	24897	25297	24681	25052	0.9620
CREMA-D	# Cases	4464	4464	4464	4464

TABLE XI : Statistics of Rule 4–7 cases, including correct validations and privacy-preservation (PP) ratios.

Fig. 8 : FFE task on DFEW dataset using R(2+1)D model without privacy preservation

Fig. 9 : FFE task on DFEW dataset using I3D model without privacy preservation

Fig. 10 : FFE task on CREMA-D dataset using R(2+1)D model without privacy preservation

Fig. 11 : FFE task on CREMA-D dataset using I3D model without privacy preservation

Fig. 12 : FFE task on RAVDESS dataset using R(2+1)D model without privacy preservation

Fig. 13 : FFE task on RAVDESS dataset using I3D model without privacy preservation

Fig. 14 : FFE task on privacy-preserved and denoised DFEW dataset using R(2+1)D model

Fig. 15 : FFE task on privacy-preserved and denoised DFEW dataset using I3D model

Fig. 16 : FFE task on privacy-preserved and denoised CREMA-D dataset using R(2+1)D model

Fig. 17 : FFE task on privacy-preserved and denoised CREMA-D dataset using I3D model

Fig. 18 : FFE task on privacy-preserved and denoised RAVDESS dataset using R(2+1)D model

Fig. 19 : FFE task on privacy-preserved and denoised RAVDESS dataset using I3D model

Facial Expression Recognition

Facial Expression Recognition studies how computers infer emotion and facial actions from images and video.

See all 79 papers

Expression-Diverse References for Identity-Preserving Video Generation
Oct 8, 2026Tianwen Fu, Wenbin Teng, Gonglin Chen +3Identity-Preserving Video Generation Video Generation Evaluation
Institute for Creative Technologies, University of Southern California
Private Face Recognition Training Dataset Publication via Identity-Decoupled and Geometry-Preserving Face Distillation
Jul 30, 2026Shuhuan Chen, Xiangyu Zhu, Weisong Zhao +6Face Recognition Knowledge Distillation
1IIE, CAS · 2CBSR&MAIS, CASIA · 4SAI, UCAS +3
Face anonymization preserving facial expressions and photometric realism
Mar 18, 2026Luigi Celona, Simone Bianco, Raimondo SchettiniPrivacy-Preserving ML
Department of Informatics, Systems and Communication, University of Milano-Bicocca, 20126 Milan, Italy

Oct 8, 2026J/K move · Enter open · S save

Expression-Diverse References for Identity-Preserving Video Generation

Tianwen Fu, Wenbin Teng, Gonglin Chen, Junyi Ouyang+2

Institute for Creative Technologies, University of Southern California

Identity-Preserving Video Generation Video Generation Evaluation

Open paper AskPDF arXiv

Identity-preserving video generation aims to maintain a subject's identity while synthesizing realistic videos. Yet a single reference portrait captures the subject's appearance under only one facial configuration. As expressions change, facial appearance can vary in highly identity-specific ways, leaving the subject's appearance under unseen expressions underdetermined by the reference alone. This expression-dependent variation also complicates evaluation: similarity to a neutral reference may decrease under strong expressions even for real images of the same person. We investigate this limitation from both generation and evaluation perspectives. First, we quantify how face-recognition similarity varies with expression intensity using controlled photographs and MEAD videos. We then construct a compact yet expressive reference gallery that captures diverse expression-dependent facial configurations. Matching against this gallery provides a more robust measure of identity similarity under expressive motion. To further expose performance degradation with expression intensity, we report identity similarity separately for mild, intense, and extreme expressions. For generation, we extend Stand-In to condition on our expression-diverse reference sets and develop a data-curation pipeline that extracts consistent yet diverse face crops from training videos. In practical settings where only a single portrait is available, we construct the reference set by synthesizing additional expressions with a pretrained facial reenactment model. On our controlled benchmark, both real and synthesized reference sets outperform the evaluated baselines in identity similarity across all three expression-intensity regimes, with the largest improvements for extreme expressions.

Abstract

Facial Expression Recognition

Explore similar work

Expression-Diverse References for Identity-Preserving Video Generation

Private Face Recognition Training Dataset Publication via Identity-Decoupled and Geometry-Preserving Face Distillation

Face anonymization preserving facial expressions and photometric realism