Beyond Classification: Task-Dependent Learnability under Privacy-Motivated Image Transformations

Leon Ranke; Wolfgang Hübner; Ronny Hug; Michael Arens; Jürgen Beyerer

cs.CVAug 27, 2026

Beyond Classification: Task-Dependent Learnability under Privacy-Motivated Image Transformations

Authors: Leon Ranke, Wolfgang Hübner, Ronny Hug, Michael Arens, Jürgen Beyerer

Organizations: Fraunhofer Institute of Optronics, System Technologies and Image Exploitation (IOSB), Gutleuthausstraße 1, 76275 Ettlingen, Germany · Karlsruhe Institute of Technology (KIT)

Abstract

Privacy-Enhancing Technologies (PETs) in computer vision often rely on noise or image perturbations to protect visual data while securely processing it, creating a trade-off between task performance and protection. This trade-off is commonly evaluated using image classification, which primarily captures semantic separability and remains robust despite significant geometric, spatial layout or local boundary alterations. As a result, it is too simplistic as a proxy for generic vision tasks. Exhaustive downstream-task evaluation, however, is computationally expensive because models must often be trained for each PET transformation and parameter setting. We therefore propose a compute-aware multi-task protocol for evaluating PETs in model training. It combines lightweight proxy tasks that target complementary aspects of visual structure while remaining simple and fast to compute. Across irreversible privacy transformations, key-based block primitives, and learnable image encryption schemes, we demonstrate that PETs with similar classification accuracy can differ substantially on other tasks. The outcomes highlight the need for PET evaluation protocols that move beyond classification-only reporting.

Figures & tables

Representation-altering scheme	Dataset(s) used to evaluate utility
Learnable Image Encryption [ 48 ]	CIFAR-10 [ 23 ]
Pixel-Based Image Encryption [ 47 ]	CIFAR-10 [ 23 ] , STL-10 [ 5 ]
Block-wise Scrambled Image Recognition [ 34 ]	CIFAR-10, CIFAR-100 [ 23 ]
InstaHide [ 18 ]	CIFAR-10, CIFAR-100 [ 23 ] , MNIST [ 28 ] , ImageNet [ 6 ] (subset)
Learnable Image Encryption for Vision Transformers [ 17 ]	CIFAR-10 [ 23 ] , Tiny-ImageNet [ 27 ]

Table 1: Datasets used to evaluate model utility in representation-altering PETs.

Name	Abbreviation	Range	Heuristic target
Perceptual Leakage
Mean Squared Error	MSE	$[0,(L-1)^{2}]$	$\frac{L^{2}-1}{6}$
Peak Signal-to-Noise Ratio (dB)	PSNR	$[0,\infty)$	$10\log_{10}\!\frac{6(L-1)}{L+1}$
Structural Similarity Index Measure [ 49 ]	SSIM	$[-1,1]$	$0$
Learned Perceptual Image Patch Similarity [ 52 ]	LPIPS	$[0,\infty)$	$\uparrow$
Mutual Information (bits)	MI	$[0,\log_{2}(L)]$	$0$

Table 2: Image-domain obfuscation and leakage indicators for RGB images. Metrics are computed per channel (R, G, B) and averaged. Intensities are in

\{0,\dots,L-1\}

, with

L=256

throughout.

Panel of Figure 1 — Figure 1: Qualitative examples of transformation families across increasing scale.

Task	Probes	Approx. Time
Classification	semantic separability	$\approx 1.4$ h
Angle Prediction	global orientation / geometric consistency	$\approx 0.8$ h
Jigsaw Puzzle Solving	spatial compatibility / local structure	$\approx 9.4$ h

Table 3: Overview of investigated proxy tasks, including approximate training time.

Panel of Figure 2 — Figure 2: Relationship between the puzzle size $k$ and the transformation block size $B$ .

Figure 3: Classification accuracy against transformation strength. Accuracy remains above chance for all transformations, indicating that semantic separability is preserved even under substantial image-domain alteration.

Figure 4: Mean angular error against transform strength (lower is better). Prediction remains robust under transformations that preserve coarse global structure, such as moderate blur, but degrades strongly when spatial order is removed at larger scales or when local and global disruptions are combined.

Figure 5: Jigsaw puzzle-solving accuracy for puzzles of different sizes; dashed coloured curves indicate evaluation with a different key from the training key. Puzzle solving exposes spatial placement cues that classification does not capture. LOI and some block-shuffling settings degrade strongly when spatial order is removed at puzzle-relevant scales, whereas several key-based transformations remain highly solvable.

Figure 6: Obfuscation-utility trade-off. Normalised task utility against normalised LPIPS. For angle prediction, the utility direction is inverted, since lower errors are better.

Hyperparameter	Value
Dataset	Animal Species Classification dataset [ 1 ]
Number of classes	$15$
Data augmentation	None
Model architecture	ResNet-34 [ 15 ]
Initial weights	Random (trained from scratch)
Input resolution	$3\times 416\times 416$

Table 4: Hyperparameters for Animal Species Classification.

Hyperparameter	Value
Dataset	Animal Species Classification dataset [ 1 ]
Angle sampling	$\alpha_{1},\alpha_{2}\sim\mathcal{U}[-180^{\circ},180^{\circ}]$
Data augmentation	None
Model architecture	CNN ( $2\times$ conv, max-pool, $4\times$ linear)
Initial weights	Random (trained from scratch)
Input resolution	$3\times 304\times 304$

Table 5: Hyperparameters for Relative Angle Prediction.

Hyperparameter	Value
Dataset	Animal Species Classification dataset [ 1 ]
Puzzle sizes $k$	$\{4,6,8\}$
Permutations	Sampled uniformly at random during training
Data augmentation	None
Model architecture	JPDVT [ 31 ]
Diffusion timesteps $T$	$1000$

Table 6: Hyperparameters for Jigsaw Puzzle Solving.

Figure 7: Obfuscation–utility trade-off: normalised task utility against normalised MSE.

Figure 8: Obfuscation–utility trade-off: normalised task utility against normalised PSNR.

Figure 9: Obfuscation–utility trade-off: normalised task utility against normalised SSIM.

Figure 10: Obfuscation–utility trade-off: normalised task utility against normalised MI.

Figure 11: Obfuscation–utility trade-off: normalised task utility against normalised

\chi^{2}

Figure 12: Obfuscation–utility trade-off: normalised task utility against normalised

D_{\mathrm{KL}}

Figure 13: Obfuscation–utility trade-off: normalised task utility against normalised IE.

Figure 14: Obfuscation–utility trade-off: normalised task utility against normalised 2D-IE.

Figure 15: Obfuscation–utility trade-off: normalised task utility against normalised CC.

Figure 16: Obfuscation–utility trade-off: normalised task utility against normalised SFM.

Privacy-Preserving ML

Privacy-preserving machine learning develops methods to learn, share, and run models while limiting exposure of sensitive data.

See all 433 papers

What Paired Evaluations Reveal under Visual Perturbations
Sep 28, 2026Yongda Wei, Chen Zhang, Yifei Wang +4Image Corruption Robustness Adversarial Robustness
Privacy-Preserving Object Detection for Vision Transformer-Based Models
Aug 21, 2026Homare Sueyoshi, Kiyoshi Nishikawa, Hitoshi KiyaPrivacy-Preserving ML Object Detection
Tokyo Metropolitan University 6-6, Asahigaoka, Hino, Tokyo, Japan
PrivacyBench: Privacy Isn't Free in Hybrid Privacy-Preserving Vision Systems
Feb 21, 2026Nnaemeka Obiefuna, Samuel Oyeneye, Similoluwa Odunaiya +2Privacy-Preserving ML Differential Privacy
1ML Collective · 2Friedrich-Alexander-Universität Erlangen-Nürnberg · 3Carnegie Mellon University

Sep 28, 2026J/K move · Enter open · S save

What Paired Evaluations Reveal under Visual Perturbations

Yongda Wei, Chen Zhang, Yifei Wang, Xinyu Wang+3

Image Corruption Robustness Adversarial Robustness

Open paper AskPDF arXiv

Robustness evaluation must examine diverse visual perturbations, while benchmarks cover only some real-world conditions and physical testing is costly. Paired evaluations link clean and perturbed predictions for the same image, capturing changes in correctness, confidence, and acceptance beyond aggregate accuracy. We investigate how this image correspondence supports two needs in robustness evaluation: interpreting paired evaluation results and prioritizing samples for physical testing. To interpret paired evaluation results, we fix both sets of prediction records and vary their correspondence within each class. We prove that classwise correct-correct counts give the same sharp bounds on lost acceptance and mean true-class probability decrease among retained-correct inputs as any feasible five-state refinement. Distinguishing persistent from changed wrong answers can further constrain accepted-error transitions, while shared correspondence can establish policy orderings left unresolved by separate cost intervals. To prioritize samples for physical testing, we retain each image's synthetic responses and rank clean-correct images by their mean true-class probability under corruption. Across 44 classifiers, testing the highest-risk 20% finds 67% and 45% of failures under mild screen and print recaptures, versus 58% and 36% for clean confidence and 60% and 37% for an equal-size natural-transformation average. With both probability averaging and an A3Rank scoring adaptation, the tested corruption set yields higher mean failure recall than the natural-transform set; differences between scores depend on the source and budget. Together, these findings show that the value of correspondence depends on the evaluation objective: classwise counts suffice for specified reliability bounds, while image-specific synthetic responses improve the allocation of physical tests within the evaluated pool.

Abstract

Privacy-Preserving ML

Explore similar work

What Paired Evaluations Reveal under Visual Perturbations

Privacy-Preserving Object Detection for Vision Transformer-Based Models

PrivacyBench: Privacy Isn't Free in Hybrid Privacy-Preserving Vision Systems