TADP: Dual-Branch Temporally Adaptive Differential Privacy for Expression-Aware Video Anonymization

Feng Xu; Xun Li; Lars Petersson; Yulei Sui; David Ahmedt-Aristizabal; Dadong Wang

cs.CVOct 30, 2026

TADP: Dual-Branch Temporally Adaptive Differential Privacy for Expression-Aware Video Anonymization

Authors: Feng Xu, Xun Li, Lars Petersson, Yulei Sui, David Ahmedt-Aristizabal, Dadong Wang

Organizations: UNSW Sydney · CSIRO, Australia

Abstract

We present TADP, a temporally adaptive differential privacy (DP) framework for inference-time privacy-preserving video generation under fixed-length clip-replacement adjacency. TADP uses a dual-branch design to extract identity and expression representations, clips and privatizes both branches with temporally adaptive Gaussian mechanisms whose noise scales depend only on preceding privatized releases within the same clip and are bounded below by history-uniform calibrated minima, and accounts for their joint release via adaptive Rényi Differential Privacy (RDP) composition. The clip-level RDP guarantee is converted to $(\varepsilon,δ)$ -differential privacy, while soft quantization and subsequent temporal processing are applied only after privatization as DP post-processing. Experiments on facial expression datasets show reduced identity leakage under evaluated attacks, usable perceptual quality and temporal stability, and high facial expression recognition utility relative to the evaluated privacy-preserving baselines. TADP formally satisfies clip-level DP and empirically provides a favorable privacy--utility trade-off under the evaluated threat models and datasets.

Figures & tables

Figure 1: TADP jointly privatizes identity and expression representations over a fixed-length clip, providing a clip-level

(\varepsilon,\delta)

-DP guarantee. All subsequent temporal modeling and decoding operate only on privatized representations.

Figure 2: Overview of the TADP framework. The pipeline contains three stages: Feature Extraction, Privacy Preservation, and Post-processing. A video is divided into fixed-length clips, and every frame is processed by an Identity Encoder and an FE Encoder to obtain identity and facial-expression (FE) representations, which are stacked into clip-level features. In Privacy Preservation, each branch is clipped and privatized by temporally adaptive Gaussian noise, and the identity and FE RDP losses are composed to obtain the reported clip-level RDP guarantee. In Post-processing, the privatized identity and FE representations are mapped by Soft Quantization with the ID Codebook and FE Codebook into soft-quantized codebook representation, followed by Fusion, Intra-clip temporal modeling, Inter-clip memory, Frame to clip aggregation, Clip to video assembly, and a Decoder that produces PP Clips and the final PP Video. All post-privatization operations use only privatized representations and public metadata, incurring no additional privacy loss.

Dataset	Parameter	Identity branch	Expression branch	Shared/clip-level
CREMA-D	Clipping percentile	95th	95th	–
	Clipping norm	$C_{id}=0.539$	$C_{expr}=0.844$	–
	Calibration clipping rate	$r_{id}^{cal}=0.05$	$r_{expr}^{cal}=0.05$	–
	Test clipping rate	$r_{id}^{test}=0.0659$	$r_{expr}^{test}=0.1617$	–
	Sensitivity bound	$2C_{id}=1.078$	$2C_{expr}=1.688$	–
	Target-loss allocation	$\rho_{id}=0.4$	$\rho_{expr}=0.6$	$\rho_{id}+\rho_{expr}=1$

Table 1: Complete dataset- and branch-specific privacy-accounting configuration.

Dataset	Length statistic	$K$	$\alpha^{\star}$	$\varepsilon_{\mathrm{video}}$
CREMA-D	Minimum	1	8	3.981
	Near mean (mean $=4.37$ )	4	4	8.510
	Maximum	10	3	14.517
RAVDESS	Minimum	4	4	6.524
	Near mean (mean $=5.19$ )	5	4	7.196
	Maximum	7	4	8.539

Table 2: Dataset source-video clip-count statistics and conservative privacy guarantees obtained by composing non-overlapping 16-frame clip releases. “Near mean” is the integer clip count nearest to the dataset mean.

Method	Full-video R3D FER Acc. $\uparrow$		Full-video TimeSformer FER Acc. $\uparrow$		Full-video VideoMAE V2 FER Acc. $\uparrow$		Frame-level PLR $\downarrow$		16-frame cPLR $\downarrow$
Method	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS
FER-only	87.043	88.675	92.683	93.032	90.860	91.944	N/A	N/A	N/A	N/A
SPAct [ 9 ]	70.963	72.548	73.576	74.815	73.978	74.722	32.573	35.921	40.322	41.111
Controlled [ 44 ]	77.955	79.551	80.153	80.958	81.021	81.944	2.088	4.983	2.220	5.833
Swapping [ 46 ]	65.194	65.918	68.985	69.112	70.054	71.111	4.023	4.856	4.194	5.556
MinusFace [ 28 ]	76.562	78.618	82.586	82.956	83.871	83.611	6.569	8.874	6.935	9.166

Table 3: Full-video FER under

K

-specific composed budgets, frame-level PLR, and 16-frame cPLR (%); see Sec. 4.1.3 .

Dataset	Method	FWE $\downarrow$	FVD $\downarrow$	tLPIPS $\downarrow$	T-ID $\uparrow$
CREMA-D	SPAct [ 9 ]	0.1023	309.5686	0.0974	0.1032
	Controlled [ 44 ]	0.1285	485.5864	0.1459	0.0481
	Swapping [ 46 ]	0.0345	295.5713	0.0348	0.7691
	MinusFace [ 28 ]	0.0586	395.5390	0.1586	0.0287
	K.P. [ 45 ]	0.0923	408.3363	0.1028	0.1785
	Pi-Net [ 7 ]	0.0358	292.4851	0.0557	0.7802

Table 4: Temporal consistency on CREMA-D and RAVDESS.

Panel of Figure 3 — Figure 3: Video temporal consistency visualization. The 64 generated frames are arranged in temporal order from left to right and top to bottom, showing that the privacy-preserved video maintains smooth frame-to-frame appearance and expression dynamics.

	Method	CREMA-D				RAVDESS
	Method	SSIM $\downarrow$	PSNR $\downarrow$	PLR $\downarrow$	cPLR $\downarrow$	SSIM $\downarrow$	PSNR $\downarrow$	PLR $\downarrow$	cPLR $\downarrow$
U-Net	SPAct [ 9 ]	0.5358	23.1459	42.890%	54.913%	0.5421	23.0751	43.861%	56.497%
	Controlled [ 44 ]	0.4283	11.8560	5.847%	10.573%	0.4641	11.9177	6.600%	11.556%
	Swapping [ 46 ]	0.5012	17.9647	38.950%	42.321%	0.5186	18.0398	39.315%	44.789%
	MinusFace [ 28 ]	0.4591	13.5457	97.395%	98.667%	0.4461	13.5287	97.477%	98.991%
	K.P. [ 45 ]	0.3959	11.8308	6.103%	9.425%	0.4512	10.314	6.813%	10.543%

Table 5: After U-Net and RVRT recovery attacks, evaluating on four metrics.

Method	Probe	CREMA-D					RAVDESS
Method	Probe	TAR@FAR= $10^{-2}$ $\downarrow$	TAR@FAR= $10^{-3}$ $\downarrow$	EER $\uparrow$	DIR@FAR= $10^{-2}$ $\downarrow$	DIR@FAR= $10^{-3}$ $\downarrow$	TAR@FAR= $10^{-2}$ $\downarrow$	TAR@FAR= $10^{-3}$ $\downarrow$	EER $\uparrow$	DIR@FAR= $10^{-2}$ $\downarrow$	DIR@FAR= $10^{-3}$ $\downarrow$
Original	Direct	94.263	88.136	3.950	$90.359\pm 1.105$	$82.490\pm 1.542$	96.058	91.227	2.836	$93.512\pm 0.913$	$87.126\pm 1.362$
SPAct	Direct	38.670	22.448	28.743	$31.592\pm 2.431$	$16.884\pm 1.920$	41.869	25.797	26.540	$34.644\pm 3.153$	$19.725\pm 2.539$
	Recovered	52.720	34.981	22.114	$45.231\pm 2.701$	$28.164\pm 2.277$	57.430	39.607	19.875	$49.336\pm 3.017$	$32.516\pm 2.740$
Controlled	Direct	6.405	2.127	42.810	$4.953\pm 0.830$	$1.573\pm 0.418$	8.213	2.877	40.622	$6.549\pm 1.193$	$2.094\pm 0.650$
	Recovered	12.871	5.310	37.615	$10.628\pm 1.373$	$3.913\pm 0.704$	15.113	6.425	35.217	$12.716\pm 1.644$	$4.819\pm 0.983$

Table 6: Open-set identity verification and identification under anonymization-aware attackers. For each anonymization method, the identity attacker is separately adapted using only privacy-preserved outputs from the development actors. All evaluation actors are unseen during attacker adaptation. “Direct” denotes privacy-preserved probes, and “Recovered” denotes probes reconstructed by the method-specific RVRT attacker. Lower TAR and DIR and higher EER indicate stronger identity privacy.

	Dataset	FID			MMD
	Dataset	O vs P	O vs R	P vs R	O vs P	O vs R	P vs R
U-Net	CREMA-D	130.5908	144.9318	72.0473	0.0308	0.0374	0.0014
U-Net	RAVDESS	412.3707	436.4749	356.8630	0.0331	0.0427	0.0057
RVRT	CREMA-D	133.6474	164.9867	81.7367	0.0325	0.0395	0.0021
RVRT	RAVDESS	454.1176	439.0419	350.8485	0.0510	0.0428	0.0052

Table 7: Comparison of identity feature distributions among original data (O), privacy-preserved data (P) and recovery-attacked data (R) under U-Net and RVRT.

Panel of Figure 4 — Figure 4: t-SNE visualization of identity features on CREMA-D (left) and RAVDESS (right). Blue, orange, and green points denote original, privacy-preserved, and recovery-attacked identity features (U-Net), respectively.

Panel of Figure 5 — Figure 5: t-SNE visualization of identity features on CREMA-D (left) and RAVDESS (right). Blue, orange, and green points denote original, privacy-preserved, and recovery-attacked identity features (RVRT), respectively.

Stage	Evaluated representation	Evaluator	CREMA-D (%)	RAVDESS (%)
Random chance	–	–	16.67	12.50
Pre-privatization	Clipped expression feature $\bar{z}^{expr}$	Linear probe	$62.03\pm 0.15\%$	$68.05\pm 0.45\%$
DP release	Privatized expression feature $\tilde{z}^{expr}$	Linear probe	$20.04\pm 0.11\%$	$19.54\pm 0.34\%$
Post-processing	Soft-quantized expression feature $z_{q}^{expr}$	Linear probe	$58.80\pm 0.06\%$	$64.17\pm 0.68\%$
Post-processing	Temporally modeled feature $\bar{h}$	Linear probe	$61.52\pm 0.09\%$	$67.04\pm 0.57\%$
Released video	Generated video $\hat{X}$	R3D	83.871	82.278

Table 8: Stage-wise expression information. TADP is frozen and FER is evaluated using temporal mean pooling followed by one linear layer. The released videos are evaluated using R3D, TimeSformer and VideoMAE V2 trained on the corresponding anonymized training videos. Results are mean

\pm

std over three seeds 42, 43, and 44. The video-level R3D, TimeSformer and VideoMAE V2 entries are single-run results reproduced from the primary evaluation and are shown only as downstream-utility references; they are not directly comparable to the latent-probe accuracies. The pre-privatization representation is included as a non-private latent reference and is never released.

Dataset	Intervention	R3D FER $\uparrow$	TimeSformer FER $\uparrow$	VideoMAE V2 FER $\uparrow$	PLR $\downarrow$	cPLR $\downarrow$
CREMA-D	Full TADP	83.871	87.634	88.011	1.237	1.452
	Expression branch shuffled	31.223	34.742	35.486	1.181	1.405
	Identity branch shuffled	80.620	84.945	85.328	1.051	1.219
	Both branches shuffled	18.407	19.118	18.831	0.989	1.104
	Expression branch set to zero	25.713	28.986	29.652	1.298	1.423
	Noise-only input	17.220	17.876	17.556	1.089	1.085

Table 9: Inference-time input-intervention analysis. All interventions are applied to the same trained TADP model. Shuffling is performed across clips, preserving the approximate marginal distribution of each representation while breaking its correspondence with the submitted clip.

Dataset	R3D FER	TimeSformer FER	VideoMAE V2 FER	Prediction agreement
CREMA-D	$83.76\pm 0.31$	$87.51\pm 0.28$	$87.92\pm 0.24$	$94.13\pm 0.97$
RAVDESS	$82.15\pm 0.44$	$88.94\pm 0.36$	$88.62\pm 0.31$	$92.67\pm 0.86$

Table 10: Utility stability over independent Gaussian releases of the same test clips. FER is measured after regenerating the complete test set for each release seed.

Dataset	Branch	Mechanism	$s_{\min}$	Observed Min	Observed Mean	Std	Observed Max
CREMA-D	ID	A6 static	8.921	8.921	8.921	0	8.921
CREMA-D	ID	TADP adaptive	8.921	8.921	9.275	0.1623	14.348
CREMA-D	Expression	A6 static	11.406	11.406	11.406	0	11.406
CREMA-D	Expression	TADP adaptive	11.406	11.406	11.891	0.1939	13.024
RAVDESS	ID	A6 static	17.088	17.088	17.088	0	17.088
RAVDESS	ID	TADP adaptive	17.088	17.088	17.667	0.2450	25.107

Table 11: Realized adaptive-scale statistics over all non-padded frames in the actor-disjoint test sets.

Dataset	Branch	Clipped-feature norm	Perturbation norm	Noise/signal ratio
		$\\|\bar{z}_{i}^{b}\\|_{2}$	$\\|s_{b,i}\xi_{b,i}\\|_{2}$	$\frac{\\|s_{b,i}\xi_{b,i}\\|_{2}}{\\|\bar{z}_{i}^{b}\\|_{2}}$
CREMA-D	ID	$0.269\pm 0.093$	$296.7\pm 8.4$	$1166.6\,[669.9,2031.4]$
CREMA-D	Expression	$0.422\pm 0.039$	$380.4\pm 10.5$	$904.9\,[772.4,1060.2]$
RAVDESS	ID	$0.341\pm 0.104$	$565.2\pm 14.8$	$1732.3\,[1058.6,2834.5]$
RAVDESS	Expression	$0.460\pm 0.052$	$500.1\pm 13.3$	$1093.7\,[904.1,1323.2]$

Table 12: Clipped-feature and Gaussian-perturbation magnitudes over all non-padded frames in the actor-disjoint test sets. Both quantities are measured before the fixed global rescaling, and their noise-to-signal ratio is invariant to this rescaling. We report mean

\pm

standard deviation for the norms and median with 5th-95th percentiles for the per-frame ratio.

Dataset	Branch	Codebook norm	Assignment entropy	Max probability	Cross-draw agreement
CREMA-D	ID	$0.98\pm 0.07$	$1.84\pm 0.31$	$0.43\pm 0.09$	0.06
CREMA-D	Expression	$1.04\pm 0.09$	$2.11\pm 0.28$	$0.37\pm 0.08$	0.09
RAVDESS	ID	$1.01\pm 0.08$	$1.76\pm 0.34$	$0.46\pm 0.10$	0.05
RAVDESS	Expression	$1.06\pm 0.10$	$2.04\pm 0.30$	$0.39\pm 0.08$	0.08

Table 13: Soft-quantization diagnostics over non-padded test frames. Assignment entropy is computed using natural logarithms; with 1,000 codebook entries, uniform assignment has entropy

\log(1000)=6.91

. Cross-draw agreement is the proportion of independent releases selecting the same top-1 codebook entry for the same frame.

Input / setting	WAR $\uparrow$	UAR $\uparrow$	Expression clipping rate $\downarrow$	tLPIPS $\downarrow$
Original DFEW videos	55.708	45.115	–	–
TADP reconstruction without DP perturbation	34.822	27.644	48.735	0.156
TADP with clip-level DP	20.912	16.839	48.761	0.211

Table 14: Cross-domain stress test on the in-the-wild DFEW dataset. Original-video performance provides a task-specific reference, while reconstruction without DP perturbation separates reconstruction and domain-shift errors from the additional degradation caused by privatization.

Variant	DP?	$\rho_{id},\rho_{expr}$	R3D: FER Acc. (%) $\uparrow$		TimeSformer: FER Acc. (%) $\uparrow$		VideoMAE V2: FER Acc. (%) $\uparrow$		PLR (%) $\downarrow$		cPLR (%) $\downarrow$		FWE $\downarrow$		FVD $\downarrow$		tLPIPS $\downarrow$		T-ID $\uparrow$
Variant	DP?	$\rho_{id},\rho_{expr}$	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS
A1	No	–	84.221	83.181	88.436	89.686	89.652	90.273	3.387	5.556	3.899	6.592	0.0245	0.0211	290.3011	251.5844	0.0138	0.0258	0.7803	0.7923
A2	Yes	0.5, 0.5	53.954	42.591	52.582	52.513	54.637	58.397	2.043	4.444	2.933	5.231	0.1056	0.0987	302.1953	275.1840	0.0992	0.1304	0.5309	0.5812
A3	Yes	same as full	24.839	25.000	27.956	26.944	26.370	26.113	1.505	4.166	2.018	4.658	0.6743	0.7941	702.5868	689.4917	0.8702	0.8910	0.0859	0.0764
A4	Yes	same as full	62.957	65.833	62.903	64.722	67.370	68.104	1.290	4.722	2.548	4.801	0.0485	0.0397	327.0942	309.8031	0.0853	0.0944	0.5428	0.6041
A5	Yes	same as full	73.226	72.778	80.108	79.722	82.138	81.643	1.559	5.000	2.354	4.737	0.0281	0.0285	287.2386	261.5593	0.0258	0.0312	0.7903	0.7544

Table 15: Ablation study results for the dual-branch mechanism.

$T$	(R3D) FER Acc. $\uparrow$		PLR $\downarrow$		Time $\downarrow$		Memory $\downarrow$
$T$	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS	CREMA-D	RAVDESS
8	84.301%	83.056%	1.290%	4.722%	21’32	4’57	58GB	62GB
16	83.871%	82.278%	1.237%	3.889%	21’45	4’59	66GB	73GB
32	53.011%	55.278%	1.129%	4.167%	22’14	5’08	78GB	85GB

Table 16: Ablation of the clip length at the same final clip-level privacy budget. Runtime and memory are measured on a single NVIDIA H100 GPU with a batch size of 2 on the test datasets.

Target $\varepsilon$	Achieved $\varepsilon$	$\rho_{id},\rho_{expr}$	$s_{id,\min},s_{expr,\min}$	FER Acc. $\uparrow$	PLR $\downarrow$	cPLR $\downarrow$	FWE $\downarrow$	FVD $\downarrow$	tLPIPS $\downarrow$	T-ID $\uparrow$
1	0.994	0.1, 0.9	69.124, 36.080	22.634%	1.129%	1.453%	0.2491	467.8613	0.6089	0.2548
2	1.985	0.2, 0.8	24.716, 19.351	25.161%	1.183%	1.937%	0.1986	368.6407	0.1201	0.4164
4	3.981	0.4, 0.6	8.921, 11.406	83.871%	1.237%	1.452%	0.0214	284.8561	0.0317	0.7937
8	7.954	0.5, 0.5	4.251, 6.656	53.954%	2.043%	2.025%	0.1056	302.1953	0.0992	0.5309

Table 17: Privacy–utility trade-off under jointly accounted dual-branch clip-level privacy budgets on CREMA-D.

Target $\varepsilon$	Achieved $\varepsilon$	$\rho_{id},\rho_{expr}$	$s_{id,\min},s_{expr,\min}$	FER Acc. $\uparrow$	PLR $\downarrow$	cPLR $\downarrow$	FWE $\downarrow$	FVD $\downarrow$	tLPIPS $\downarrow$	T-ID $\uparrow$
1	0.991	0.1, 0.9	87.160, 39.466	25.556%	4.167%	4.491%	0.6104	385.6503	0.8045	0.2108
2	1.983	0.3, 0.7	25.406, 22.593	28.333%	4.444%	4.821%	0.0511	329.0132	0.1458	0.5311
3	2.988	0.3, 0.7	17.088, 15.197	82.278%	3.889%	4.444%	0.0203	249.5107	0.0414	0.8032
8	7.962	0.5, 0.5	5.342, 7.256	43.889%	5.000%	6.333%	0.0704	333.6845	0.0961	0.6853

Table 18: Privacy–utility trade-off under jointly accounted dual-branch clip-level privacy budgets on RAVDESS.

Rényi Differential Privacy

See all 9 papers

Knowledge Priors for Identity-Disentangled Open-Set Privacy-Preserving Video FER
Mar 22, 2026Feng Xu, Xun Li, Lars Petersson +3Facial Expression Recognition Privacy-Preserving ML
School of CSE, UNSW Sydney, Australia · CSIRO’s Data61, Australia
Delving into the Temporal Challenges of Unified Video Protection Against Image-to-Video and Fine-Tuning-based Customization
Jul 14, 2026Yuxin Huang, Ziming Hong, Mingming Gong +3Video Diffusion Models Adversarial Robustness
1Sydney AI Centre, The University of Sydney · University of Melbourne · 5Mohamed bin Zayed University of Artificial Intelligence +2
Private and Stable Test-Time Adaptation with Differential Privacy
Jun 1, 2026Zefeng Li, Qiaoyue Tang, Mathias Lecuyer +1Continual Test-Time Adaptation Privacy-Preserving ML
University of British Columbia · Vector Institute

Mar 22, 2026J/K move · Enter open · S save

Knowledge Priors for Identity-Disentangled Open-Set Privacy-Preserving Video FER

Feng Xu, Xun Li, Lars Petersson, Yulei Sui+2

School of CSE, UNSW Sydney, Australia · CSIRO’s Data61, Australia

Facial Expression Recognition Privacy-Preserving ML Representation Disentanglement

Open paper AskPDF arXiv

Facial expression recognition relies on facial data that inherently expose identity and thus raise significant privacy concerns. Current privacy-preserving methods typically fail in realistic open-set video settings where identities are unknown, and identity labels are unavailable. We propose a two-stage framework for video-based privacy-preserving FER in challenging open-set settings that requires no identity labels at any stage. To decouple privacy and utility, we first train an identity-suppression network using intra- and inter-video knowledge priors derived from real-world videos without identity labels. This network anonymizes identity while preserving expressive cues. A subsequent denoising module restores expression-related information and helps recover FER performance. Furthermore, we introduce a falsification-based validation method that uses recognition priors to rigorously evaluate privacy robustness without requiring annotated identity labels. Experiments on three video datasets demonstrate that our method effectively protects privacy while maintaining FER accuracy comparable to identity-supervised baselines.

Abstract

Rényi Differential Privacy

Explore similar work

Knowledge Priors for Identity-Disentangled Open-Set Privacy-Preserving Video FER

Delving into the Temporal Challenges of Unified Video Protection Against Image-to-Video and Fine-Tuning-based Customization

Private and Stable Test-Time Adaptation with Differential Privacy