Recent advances in large-scale video models have significantly improved video understanding across domains such as surveillance, healthcare, and entertainment. However, these models also amplify privacy risks by encoding sensitive attributes, including facial identity, race, and gender. While image anonymization has been extensively studied, video anonymization remains relatively underexplored, even though modern video models can leverage spatiotemporal motion patterns as biometric identifiers. To address this challenge, we propose a novel attention-driven spatiotemporal video anonymization framework based on systematic disentanglement of utility and privacy features. Our key insight is that attention mechanisms in Vision Transformers (ViTs) can be explicitly structured to separate action-relevant information from privacy-sensitive content. Building on this insight, we introduce two task-specific classification tokens, an action CLS token and a privacy CLS token, that learn complementary representations within a shared Transformer backbone. We contrast their attention distributions to compute a utility-privacy score for each spatiotemporal tubelet, and keep the top-k tubelets with the highest scores. This selectively prunes tubelets dominated by privacy cues while preserving those most critical for action recognition. Extensive experiments demonstrate that our approach maintains action recognition performance comparable to models trained on raw videos, while substantially reducing privacy leakage. These results indicate that attention-driven spatiotemporal pruning offers an effective and principled solution for privacy-preserving video analytics.
Figures & tables
Figure 1: Overview of our proposed video anonymization framework. A ViT-based anonymizer fA employs two independent CLS tokens, act_CLS and priv_CLS , to attend all tubelet sequence. Each tubelet i is assigned a utility-privacy score si=αiact−λpriv⋅αipriv , where αiact and αipriv denote the CLS-to-tubelet attention weights. Tubelets are then ranked by si and the top- k tubelets are retained to suppress privacy-sensitive content while preserving action-relevant cues, while the remaining low-score tubelets are compressed via a token fusion module. The anonymized video is then fed to a utility branch fT and a budget model fB under action and privacy objectives.
Method
VPUCF
VPHMDB
PAHMDB
Action
Privacy
Action
Privacy
Action
Privacy
Top-1 ( ↑% )
cMAP ( ↓% )
F1 ( ↓ )
Top-1 ( ↑% )
cMAP ( ↓% )
F1 ( ↓ )
Top-1 ( ↑% )
cMAP ( ↓% )
F1 ( ↓ )
Raw data
84.88
76.62
0.684
81.05
76.62
0.684
65.24
70.20
0.396
Downsample-2 ×
54.11
57.31
0.512
40.80
71.35
0.601
36.10
61.20
0.111
Downsample-4 ×
39.65
52.61
0.493
31.32
69.79
0.594
25.80
41.40
0.081
Blackening
53.13
56.39
0.457
38.27
74.06
0.649
34.20
63.80
0.386
Table 1: Comparison of different privacy-preserving methods on known action and known private attributes. ↓ % is the relative drop from the raw data, and – indicates the model does not perform the experiment on the dataset. High performance in action and low performance in privacy are considered as better. Best results are in bold.
Method
VPUCF → VPHMDB
VPHMDB → VPUCF
Action
Privacy
Action
Privacy
Top-1 ( ↑ %)
cMAP ( ↓ %)
F1 ( ↓ )
Top-1 ( ↑ %)
cMAP ( ↓ %)
F1 ( ↓ )
Raw data
52.31
75.58
0.673
92.23
76.62
0.701
VITA [ 29 ]
—
—
—
77.48 ( ↓ 14.75 )
76.02 ( ↓ 0.60 )
0.669 ( ↓ 0.032 )
SPAct [ 8 ]
—
—
—
78.13 ( ↓ 14.10 )
75.98 ( ↓ 0.64 )
0.661 ( ↓ 0.040 )
STPrivacy [ 17 ]
49.56 ( ↓ 2.76 )
71.25 ( ↓ 4.33 )
0.595 ( ↓ 0.078 )
81.04 ( ↓ 11.19 )
74.60 ( ↓ 2.22 )
0.645 ( ↓ 0.056 )
Table 2: Comparison of different privacy-preserving methods on novel actions and novel private attributes datasets. ↓ % represents the relative drop from raw data, and – indicates the model does not perform the experiment on the dataset. High performance in action and low performance of privacy is considered as better. Best results are in bold.
Figure 2: Visualization of our framework on three actions: (a) weightlifting, (b) YoYo, and (c) playing violin. For each ten frame clip, the top row shows raw consecutive frames, and the bottom row shows the corresponding anonymized frames produced by our method.
Keep rate r
Top-1 (%)
cMAP (%)
1.0
81.05
76.62
0.9
79.59 ( ↓ 1.46)
70.32 ( ↓ 6.30)
0.8
69.03 ( ↓ 12.02)
60.32 ( ↓ 16.30)
0.7
60.08 ( ↓ 20.97)
55.81 ( ↓ 20.81)
Table 3: Performance under different keep rates for the tubelet selection on VPHMDB dataset. ↓ denotes the absolute drop relative to the raw setting (when the keep rate r=1.0 ), i.e . , without pruning.
Federated video action recognition enables collaborative model training without sharing raw video data, yet remains vulnerable to two key challenges: model exposure and communication overhead. Gradients exchanged between clients and the server can leak private motion patterns, while full-model synchronization of high-dimensional video networks causes significant bandwidth and communication costs. To address these issues, we propose Federated Differential Privacy with Selective Tuning and Efficient Communication for Action Recognition, namely FedDP-STECAR. Our FedDP-STECAR framework selectively fine-tunes and perturbs only a small subset of task-relevant layers under Differential Privacy (DP), reducing the surface of information leakage while preserving temporal coherence in video features. By transmitting only the tuned layers during aggregation, communication traffic is reduced by over 99% compared to full-model updates. Experiments on the UCF-101 dataset using the MViT-B-16x4 transformer show that FedDP-STECAR achieves up to 70.2% higher accuracy under strict privacy (ε=0.65) in centralized settings and 48% faster training with 73.1% accuracy in federated setups, enabling scalable and privacy-preserving video action recognition. Code available at https://github.com/izakariyya/mvit-federated-videodp
Idris Zakariyya, Pai Chet Ng, Kaushik Bhargav Sivangi +3
School of Computing Science, University of Glasgow, G12 8QQ, Glasgow, UK · Infocomm Technology Cluster, Singapore Institute of Technology, Singapore · The Edward S. Rogers Sr. Dept. of Electrical and Computer Engineering, University of Toronto, Canada
Facial expression recognition relies on facial data that inherently expose identity and thus raise significant privacy concerns. Current privacy-preserving methods typically fail in realistic open-set video settings where identities are unknown, and identity labels are unavailable. We propose a two-stage framework for video-based privacy-preserving FER in challenging open-set settings that requires no identity labels at any stage. To decouple privacy and utility, we first train an identity-suppression network using intra- and inter-video knowledge priors derived from real-world videos without identity labels. This network anonymizes identity while preserving expressive cues. A subsequent denoising module restores expression-related information and helps recover FER performance. Furthermore, we introduce a falsification-based validation method that uses recognition priors to rigorously evaluate privacy robustness without requiring annotated identity labels. Experiments on three video datasets demonstrate that our method effectively protects privacy while maintaining FER accuracy comparable to identity-supervised baselines.
Feng Xu, Xun Li, Lars Petersson +3
School of CSE, UNSW Sydney, Australia · CSIRO’s Data61, Australia
Privacy-preserving action recognition (PPAR) enables machines to understand human activities in videos without revealing sensitive visual content. Among the various strategies for PPAR, encryption-based methods achieve strong privacy protection while maintaining high recognition performance. However, these methods lead to a catastrophic decrease in recognition performance and visual quality when the encrypted videos are compressed. That is, the previous methods are not compression-friendly. To address these issues, in this paper, we propose the first compression-friendly encryption method for PPAR, called CFE-PPAR. In CFE-PPAR, videos encrypted with secret keys can be directly recognized by a video transformer, which uses parameters transformed by the same keys as those used for video encryption. In experiments, it is verified that CFE-PPAR outperforms previous methods on the UCF101 and HMDB51 datasets under Motion-JPEG and H.264 compression.
Haiwei Lin, Shoko Imaizumi, Hitoshi Kiya
Graduate School of Informatics Chiba University Chiba, Japan · Faculty of System Design Tokyo Metropolitan University Tokyo, Japan