PrivHAR-Bench: A Graduated Privacy Benchmark Dataset for Video-Based Action Recognition

Samar Ansari

cs.CVApr 1, 2026

PrivHAR-Bench: A Graduated Privacy Benchmark Dataset for Video-Based Action Recognition

Authors: Samar Ansari

Organizations: School of Computer and Engineering Sciences University of Chester Chester, CH1 4BJ, United Kingdom

Abstract

Existing research on privacy-preserving Human Activity Recognition (HAR) typically evaluates methods against a binary paradigm: clear video versus a single privacy transformation. This limits cross-method comparability and obscures the nuanced relationship between privacy strength and recognition utility. We introduce PrivHAR-Bench, a multi-tier benchmark dataset designed to standardize the evaluation of the Privacy-Utility Trade-off in video-based action recognition. PrivHAR-Bench applies a graduated spectrum of visual privacy transformations: from lightweight spatial obfuscation to cryptographic block permutation, to a curated subset of 15 activity classes selected for human articulation diversity. Each of the 1,932 source videos is distributed across 9 parallel tiers of increasing privacy strength, with additional background-removed variants to isolate the contribution of human motion features from contextual scene bias. We provide lossless frame sequences, per-frame bounding boxes, estimated pose keypoints with joint-level confidence scores, standardized group-based train/test splits, and an evaluation toolkit computing recognition accuracy and privacy metrics. Empirical validation using R3D-18 demonstrates a measurable and interpretable degradation curve across tiers, with within-tier accuracy declining from 88.8% (clear) to 53.5% (encrypted, background-removed) and cross-domain accuracy collapsing to 4.8%, establishing PrivHAR-Bench as a controlled benchmark for comparing privacy-preserving HAR methods under standardized conditions. The dataset, generation pipeline, and evaluation code are publicly available.

Figures & tables

Figure 1: A single frame from the PrivHAR-Bench dataset shown across selected privacy tiers. From left to right: Original (no privacy), Tier 1 Blur ( $\sigma=15$ ), Tier 2 Edge (Canny), Tier 3 AES scramble at block sizes $B=4$ , $B=8$ , and $B=16$ , and the background-removed (NoBG) variant of $B=8$ . As tier level increases, spatial identity features are progressively destroyed while gross motion structure is preserved at varying degrees.

Dataset	Year	Classes	Clips	Privacy Tiers	NoBG	Lossless	Eval Toolkit
UCF101 [ 1 ]	2012	101	13,320	0	×	×	×
HMDB51 [ 36 ]	2011	51	6,766	0	×	×	×
PA-HMDB51 [ 15 ]	2020	51	515	1	×	×	×
Kinetics-400 [ 2 ]	2017	400	306K	0	×	×	×
NTU RGB+D [ 3 ]	2016	60	56,880	0	×	✓	×
NTU RGB+D 120 [ 37 ]	2019	120	114,480	0	×	✓	×

Table 1: Comparison of PrivHAR-Bench with existing datasets used for privacy-preserving HAR evaluation. PrivHAR-Bench is, to our knowledge, the first benchmark providing multiple graduated privacy tiers, background-removed variants, and a standardized evaluation toolkit within a single release.

Figure 2: The PrivHAR-Bench privacy spectrum. Each tier progressively destroys a different category of visual information, from fine-grained appearance (Tier 1) through structural contour (Tier 2) to spatial pixel arrangement (Tier 3). Tier 3 is further parameterized by block size $B$ .

Figure 3: Illustration of context bias control. Left: Tier 3 ( $B=8$ ) with background preserved, a model could exploit environmental cues. Right: the NoBG variant of the same frame, isolating the scrambled human region.

Category	Class	Clips	Groups	Med. Det. Rate
ADL / Care	BrushingTeeth	131	25	100.0%
	Haircut	130	25	99.5%
	MoppingFloor	110	25	99.7%
	ApplyEyeMakeup	145	25	100.0%
	BabyCrawling	132	25	97.5%
	ShavingBeard	161	25	100.0%

Table 2: The 15 selected classes, grouped by motion category. Clip count and group count are from the UCF101 source. Detection rate is the median per-frame person detection rate across all clips in the class using YOLOv8n-Pose.

Figure 4: Overview of the PrivHAR-Bench generation pipeline. Each source video passes through a shared ROI detection stage, after which all privacy tiers are generated in parallel using the same bounding box and mask.

Property	Value
Source dataset	UCF101 [ 1 ]
Number of classes	15
Total source clips	1,932
Frames per clip	32
Frame resolution	$224\times 224$
Number of tiers	9 (Original + Blur + Edge + B4 + B8 + B16 + 3 $\times$ NoBG)

Table 3: PrivHAR-Bench dataset composition and storage requirements.

Config	Train Tier	Eval Tier	Purpose
A	Same as Eval	Each tier	Within-tier accuracy
B	Original (Clear)	Each tier	Cross-domain generalization
C	Tier3_B8_NoBG	Tier3_B8_NoBG	Context-free recognition

Table 4: Required evaluation configurations for PrivHAR-Bench benchmark submissions. “Train Tier” indicates the privacy level of training data; “Eval Tier” indicates the privacy level of test data. Config A measures within-tier performance; Config B measures generalization from clear training data to encrypted test data.

Model	Original	Blur	Edge	B16	B8	B4	B8-NoBG
R3D-18	88.8	66.5	66.2	66.2	64.4	63.1	53.5
$\Delta_{\text{acc}}$	—	22.3	22.6	22.6	24.4	25.7	35.3

Table 5: Top-1 accuracy (%) of the R3D-18 baseline across PrivHAR-Bench privacy tiers under Config A (train and test on the same tier). Accuracy generally decreases with increasing privacy strength. The NoBG variant isolates the contribution of the scrambled human region from background context.

Model	Blur	Edge	B16	B8	B4	B8-NoBG
R3D-18	48.1	6.9	39.2	34.4	35.6	4.8

Table 6: Top-1 accuracy (%) under Config B: model trained on Original (clear) data and evaluated on each privacy-transformed tier without fine-tuning. This measures out-of-distribution robustness. Random chance for 15 classes is 6.7%.

Figure 5: Top-1 accuracy as a function of privacy tier for the R3D-18 baseline. Solid line: Config A (within-tier training). Dashed line: Config B (cross-domain, trained on Original only). The divergence between curves quantifies the domain gap introduced by each transformation. Note: The B8-NoBG condition combines two independent manipulations: cryptographic block scrambling (privacy) and background removal (context bias control). It is plotted at the rightmost position for visual continuity, but it is not strictly a point on the same linear privacy scale as the preceding tiers; it additionally removes an orthogonal confound (see Section 3.2 and Figure 3).

Model	Tier3-B8 (With BG)	Tier3-B8 (NoBG)	$\Delta$
R3D-18 (Config A)	64.4%	53.5%	$-$ 10.9 pp
R3D-18 (Config B)	34.4%	4.8%	$-$ 29.6 pp

Table 7: Effect of background removal on Tier 3 (

B=8

) recognition accuracy. The 10.9 percentage point gap confirms that the model derives substantial classification signal from environmental context [ 18 , 19 ] .

Block Size $B$	Det.	Fail Rate	Mean Sim.	Median Sim.	$<$ 0.2
2	56	88.8%	0.631	0.912	27%
4	54	89.2%	0.660	0.931	24%
8	53	89.4%	0.658	0.955	24%
16	56	88.8%	0.684	0.965	23%
32	69	86.2%	0.670	0.967	22%

Table 8: ArcFace face detection and identity verification results after block scrambling. “Det.” is the number of faces (out of 500) where ArcFace could detect a face post-scrambling. “Fail Rate” is the percentage of faces rendered undetectable. Conditional similarity is computed only over the detected subset.

Figure 6: ArcFace [ 40 ] cosine similarity between original and scrambled face crops as a function of block size

B

, computed over the subset of faces that remained detectable post-scrambling. The dashed lines indicate verification thresholds at 0.2 (conservative) and 0.3 (standard). Approximately 89% of faces are not represented in this plot because they became undetectable after scrambling.

Tier	ROI-SSIM $\downarrow$	ROI-PSNR $\downarrow$	Face Fail % $\uparrow$	Acc % $\uparrow$	$PU$ $\uparrow$
Original	1.000	$\infty$	0%	88.8	—
Tier1 (Blur)	0.764	34.88	—	66.5	0.177
Tier2 (Edge)	0.043	6.50	—	66.2	0.713
Tier3a ( $B=16$ )	0.681	35.04	88.8%	66.2	0.238
Tier3b ( $B=8$ )	0.624	32.01	89.4%	64.4	0.273
Tier3c ( $B=4$ )	0.588	30.38	89.2%	63.1	0.293

Table 9: Privacy and utility metrics across tiers. SSIM and PSNR are computed within the ROI bounding box only. Face Detection Failure Rate is the percentage of originally-detectable faces that become undetectable after transformation. Face Detection Failure Rate is reported only for Tier 3 variants, where block scrambling is applied; Tier 1 and Tier 2 are marked with dashes because the ArcFace evaluation (Section 6.3 ) was conducted exclusively on scrambled outputs.

PU

score is the composite Privacy-Utility metric (higher is better). All metrics are computed over 7,728 sampled frames.

Figure 7: Privacy-Utility space for the R3D-18 baseline on PrivHAR-Bench. Each point represents one tier under Config A. The x-axis measures privacy strength (

1-\text{SSIM}

; higher is more private); the y-axis measures recognition utility. The top-right quadrant represents the ideal operating region: high privacy with high accuracy. Tier 2 (Edge) occupies the most favorable position.

Explore similar work

Privacy-Preserving Action Recognition: Taxonomy, Methods, and Privacy-Utility Trade-offs
Aug 5, 2026Sareer Ul Amin, Muhammad Ayaz, Muhammad Munsif +1Privacy-Preserving ML Privacy-Utility Tradeoff
Ulsan National Institute of Science and Technology (UNIST), Ulsan 44919, Republic of Korea · College of Art and Technology, Chung-Ang University, Anseong, 17546, South Korea
Lens Privacy Sealing: A New Benchmark and Method for Physical Privacy-Preserving Action Recognition
May 19, 2026Mengyuan Liu, Ziyi Wang, Peiming Li +1Video Action Recognition Privacy-Preserving ML
State Key Laboratory of General Artificial Intelligence, Peking University, Shenzhen Graduate School · Department of Computer Science and Engineering, State University of New York at Buffalo
Identifying Ethical Biases in Action Recognition Models
Apr 20, 2026Ana Baltaretu, Pascal Benschop, Jan van GemertHuman Activity Recognition Algorithmic Fairness
1EEMCS, Delft University of Technology, The Netherlands

Aug 5, 2026J/K move · Enter open · S save

Privacy-Preserving Action Recognition: Taxonomy, Methods, and Privacy-Utility Trade-offs

Sareer Ul Amin, Muhammad Ayaz, Muhammad Munsif, Sanghyun Seo

Ulsan National Institute of Science and Technology (UNIST), Ulsan 44919, Republic of Korea · College of Art and Technology, Chung-Ang University, Anseong, 17546, South Korea

Privacy-Preserving ML Privacy-Utility Tradeoff

Open paper AskPDF arXiv

Video surveillance in public safety, healthcare, and smart environments has made continuous human monitoring routine, raising real risks to personal identity and appearance. Privacy-preserving action recognition (PPAR) tackles the tension between the utility of video understanding and this exposure, and has drawn fast-growing interest. However, existing surveys remain narrow. Most catalog a single mechanism family, predate recent adversarial and hybrid work, or barely address evaluation. The result is a fragmented literature with incompatible threat models, inconsistent metrics, and no shared evaluation standard. We address this with a PRISMA-guided review of 32 peer-reviewed papers (2018--2026) drawn from 885 screened records. Methods sort into five families, namely adversarial learning (52%), skeleton-based (20%), cryptographic (12%), differential privacy (8%), and hybrid (8%), each with distinct privacy, utility, and efficiency trade-offs. Evaluation is the weak point. Only 10% of papers adopt a formal privacy definition, 65% rely on ad-hoc metrics, and 40% report an inconsistently defined cMAP. The trade-offs are steep. Skeleton methods reach about 85% accuracy but drop appearance, adversarial methods hold near 80% utility at moderate privacy (cMAP 0.9 to 0.3--0.5), and differential privacy often falls below 70%. Harder conditions stay under-tested, with fewer than 15% of papers checking cross-dataset generalization, under 10% testing adaptive attackers, and real-time edge deployment nearly untouched. We contribute a two-dimensional privacy-space taxonomy, a formal threat model, a comparative trade-off analysis, the PPAR Unified Evaluation Protocol, and a roadmap centered on benchmark standardization. With this grounding, we argue PPAR can move from prototypes toward deployment, with lessons extending to face recognition and medical imaging.

Abstract

Explore similar work

Privacy-Preserving Action Recognition: Taxonomy, Methods, and Privacy-Utility Trade-offs

Lens Privacy Sealing: A New Benchmark and Method for Physical Privacy-Preserving Action Recognition

Identifying Ethical Biases in Action Recognition Models